Posts Tagged ‘updating’

Benefits: Administrative Staff

Posted on January 14th, 2011 by Nick Jackson

Behind the academic side of the University lies an army of support and administrative staff. It’s their job to look after the bits and pieces which let the University actually function, and Total ReCal was built not only to help students and academic staff get on with their life but also to make the business of supporting them easier.

First of all, Total ReCal helps staff by making any updates to calendar information replicate around the system as rapidly as possible. Since our Nucleus events platform serves as a single source of information any updates directly on Nucleus are instantaneously reflected in views on the data, and even changes to imported data such as timetables and assessments are shown a lot faster than before. A reduced lag time between making a change and systems updating means that it’s easier to make changes and cancellations to events, and that more people are likely to be informed of those changes. For some alterations we even have change detection which can be hooked into notification systems such as text, making cancelling a lecture and notifying students a simple operation.

There’s also a related benefit to the rapid updating of information in that Total ReCal (or, more accurately, Nucleus) represents a single location for calendaring data, meaning that it’s a lot easier to draw on collated data. This may initially seem like a somewhat ‘fluffy’ feature which will never be used, but with a little bit of thought it’s easy to see how it can help drive decisions. For example, we can draw pretty graphs of room usage over time and spot peaks and troughs, enabling smarter timetabling. We can detect collisions across disparate systems, reducing confusion over resource allocation. We can monitor assessment ‘pile-up’ to help spread the workload of students more evenly. In short we gain the ability to draw up reports on just about anything in real-time.

Finally – and most importantly – Total ReCal will make it easier to do things the ‘right’ way (ie by managing events centrally) rather than by groups or departments going off and doing their own thing. Where in the past things such as induction timetables were managed by departments and distributed (literally) as a gigantic Word document it’s now easier for everybody involved to just use Total ReCal and to distribute the result over My Calendar. Students can be forcibly added to calendars such as induction or exams, and our code takes care of all the hard work of making sure your events don’t collide with another. Even better, there’s no real deadline for content creation because there’s no publishing deadline. Change the event centrally and the change ripples out to all the users and other systems relying on calendar data within minutes.

We hope that within a year Total ReCal will have prompted students to demand that their departments use centralised timetabling and assessment deadline management, leading to a more unified, reliable, easy to use and just better looking life.

How we make things faster.

Posted on October 18th, 2010 by Nick Jackson

Today we’ve been playing around with our timetable parser to Nucleus connection and trying to work out why we were taking a projected 19 days to finish up parsing and inserting.

This was a problem of many, many parts. First up was Alex’s code, which was performing an update to the event on Nucleus for each one of the 1.76 million lines associating students with events. Great fun, since Total ReCal communicates with Nucleus over HTTP and our poor Apache server was melting. This was solved by using an intermediate table into which we could dump the 1.76 million lines (along with some extra data we’d generated, such as event IDs) and then read them back out again in the right order to make the inserts tidier. This reduced the number of calls to about 46500, a mere 2% of the number of things to do.

Next, we ran into an interesting problem inserting the events. The whole thing would go really quite fast until we’d inserted around 48 events, at which point it would drop to one insertion a second. Solving this involved sticking a few benchmark timers in our code to work out where the delay was happening, and after much probing it was discovered that the unique ID generation code I’d created couldn’t cope with the volume of queries, and (since it was time based) was running out of available ID numbers and having to keep running through its loop until it found a new one, taking around a second a line. Changing this to use PHP’s uniqid() function solved that little flaw by making the identifier a bit longer, meaning that the chance of a collision is now really, really small.

At the moment we’re running at about 33 inserts a second, meaning the complete inserting and updating of our entire timetable (at least the centrally managed one, the AAD faculty are off in their own little world) is done in a little over 20 minutes. We’ve had to turn off a couple of security checks, but even with these enabled the time little more than doubles and we’re currently not making use of any kind of caching on those checks (so we can get it back down again). There are also lots of other optimisations left to do.

A bit of quick number crunching reveals to me that we’re now running the process in a mere 0.08% of our original 19 days. Not bad.