All entries for April 2004

About me

Response to prompt "About me" (View all responses or all Cubicle 23 responses)

•My life and interests (in brief!)

I'm 36, and since I've been full-time in IT since 18, that's exactly half my life. Interested in new technology, buddhism, the craft of writing, people, cars and other stuff.

•What inspire and motivates me

To work? Having a nice house and car and being able to travel to far-off lands in a reasonable degree of luxury go a long way.

•My values

183, 79, 36, orange.

•Why the university is lucky to have me

I'm in the top 1% for cleverness and a hard worker, what more could the uni possibly want?

: 29 Apr 2004 14:10 |  Comments (0) |  Close comments |  Report a problem

Speeding up databases

Well, I'm feeling pretty accomplished. I spent the whole of Monday afternoon refactoring the LoadDeliveries class and managed to speed it up from 200 minutes to 10 minutes (as well as making the code far more readable). I used the following techniques to speed it up:

• Order SQL queries so that objects needed to process one record will usually also be the ones needed to process the next, ensuring a much higher JDO cache hit rate
• Maintain a cache of references to JDO objects which will be neede frequently so that they don't have to be looked up
• Keep SQL prepared statements in a wider scope than just the method where they're used – I think frequent recompiling of my prepared statements was taking up a huge amount of the time
• Don't query an extent when you have an object which has a collection which includes the object you're looking for
• Don't process data unneccessarily – now LoadDeliveries only processes data for the current academic year, or next year if a special flag is set (which we'll do once a week or some such); all other years are ignored.

LoadStudents is also taking hours, and as well as refactoring and speeding up also needs changes to cope with the academic year roll-forward in SITS. That's what I'll be working for the next couple of days. If I can get a speedup factor of 5, let alone 20, I'll be pleased.

: 27 Apr 2004 12:30 |  Comments (0) |  Close comments |  Report a problem

Day One

Created blog, partly to avoid the tedium of optimising SQL load processes.

Investigated a couple of queries about module registrations from the Academic Office. One student who claimed to have changed their modules, one where the status in OMR was 'uploaded' but there was no record in SMO. In both cases, there looks to be no problem in the OMR audit log. It's a bit unsatisfying to point the finger at SITS just because OMR keeps and audit trail and SITS doesn't, but there seems no alternative. Decided to improve the OMR audit logging even further so that there is even less doubt as to exactly what the OMR system did and when.

: 26 Apr 2004 14:17 |  Comments (0) |  Close comments |  Report a problem