Using RT at OUCS

Request Tracker is an open source issue ticketing software which we use extensively in OUCS, and Sysdev are responsible for managing it. In recent weeks, I found myself discussing some of the ways we use RT with people from the RT user and developer community, and I thought it might be interesting to share some details here.

I’m not going to provide a review or detailed description of RT as such, but the basis is a database-driven application which creates and manages tickets (the basic unit which which is analagous to a bug report, user enquiry, incident, as you prefer). The primary interface for users is incoming and outgoing email, and a web interface is provided to privileged (staff) users of the system. (It also offers a self-service web interface to end-users, which we haven’t deployed.)

RT has been around for a relatively long time, and OUCS began using it relatively early, too. The first ticket still in our main RT installation dates from September 2001, not far off 10 years ago. Initially, RT was deployed as a way for the help desk to track incoming enquiries, but since then its use has grown to all sorts of other areas, with various systems teams including ourselves using it for internal incident management, and so on. RT now handles a fairly significant volume of email, critical to OUCS’s business (providing IT services to the University).

When I first started working at OUCS, the RT system was already well-established, using RT 2, which was by that point fairly long in the tooth (the first release of RT 3 was four years previously in 2003). It had already handled more then a million tickets.

As RT is implemented in Perl, and Sysdev has always been fairly comfortable with hacking Perl, various customisations were implemented to handle specific requirements. Unfortunately, those changes were not in general made in collaboration with the developer community, largely because RT had moved on so much (some “interesting” hacks were made in order to overcome some severe performance problems, for example). When we finally managed to get time to bring our installation up to date around two years ago, a fair bit of software archaeology was required to find out what changes we had, and why!

One of the pleasant side-effects of Sysdev working on RT has been the chance to work on, and improve the packaging of RT in Debian (our Linux distribution of choice for deploying services on). Since I am a registered Debian Developer, I was in a good position to join the packaging team in Debian which had suffered from lack of sustained interest. This provided an excellent excuse to get stuck into updating the RT packages to 3.8, which I did, based on preparatory work by others. Interestingly, I wasn’t the first Sysdev team member to be involved in RT packaging for Debian; inspection of the older changelogs reveals work done by one of my predecessors. Being involved in the official packaging project has also meant that I’ve become reasonably familiar with RT4, which has been out for a few months now, even though we’re a little way off deploying that at OUCS.

Once we started to analyse how an upgrade to RT 3.8 would look at OUCS, and discussed the system with others around the department, we found that many of the tweaks added either weren’t important, or had been superceded by other changes in RT, but there were still quite a few customisations required. Since the migration, I’ve tried hard to make sure that issues we face are tracked in the upstream bug tracking system, and that patches we write are both manageable (so that they don’t present such an obstacle to future upgrades) and where possible, general enough to be fed back upstream. Inevitably this doesn’t always happen, of course — at the time of writing, we have 13 patches to the core (on top of 18 applied by Debian). These cover trivial things like adding extra convenience links to ticket display pages, to bugfixes which only arise because of some of the “interesting” ways we use RT, to a patch set originally taken from the “QueueDeactivatedScrips” extension.

One example of the “interesting” ways we use RT is in our handling of bounces; RT3, unlike RT2, doesn’t by default support routing bounce messages relating to tickets back into RT, but because of the range of internal users of the system, we needed to be able to continue to support this functionality. What we’ve ended up with is an exim filter rule, together with a scrip based on this one from the RT wiki (another very useful community resource) with some additions which email our staff users to let them know about the incoming bounce, and with a patch to ensure that bounces get routed to the right place.

It’s worth mentioning that RT supports quite a powerful overlays system which allows for plugins to cleanly implement a certain amount of new functionality without having to patch the core codebase. Some of our work is done this way, but a few more changes have been made directly, as discussed above. Nowadays we have much better ways for managing patchsets on top of official Debian packages, which makes this a less worrying prospect than it was in our RT2 days.

Our migration project also involved a fair bit of work to get the rt2tort3 scripts up and running with RT3.8 (I suspect we were one of the first to do so — after all, most people hopefully got round to the migration long ago!). That work is available as a forked github project.

The nice thing about working on RT at OUCS is that it’s an important tool used by a significant proportion of OUCS staff for their daily work, all day every day. This results in a large number of detailed change requests, which we have been diligently collating and assessing for implementation. Due to other workloads we haven’t been as responsive to those requests as we’d like, but a team internal ‘hackfest’ is planned for next year in order to hopefully address some of those (and probably also to move to RT4).

Another nice thing about RT is working with a great bunch of individuals both at Best Practical, who are the core maintainers of RT, and the external user and developer communities, both on IRC, mailing lists and wikis. This has made it especially rewarding working on the official Debian packages, as well as deploying RT here at OUCS. It’s an excellent example of Free/Libre Software working for all parties, with the users contributing valuable code back to the project, and I’m pleased to be able to participate in the free exchange of ideas and code, with the support of the OUCS OSS policy, which makes releasing code under Free/Open licences fairly trouble-free.

Coincidentally, I was able to meet up with the Best Practical team in Cambridge, MA, USA last year whilst attending the MIT Kerberos Conference; those sort of face-to-face meetings with collaborators in free software are an excellent way of getting to know the people at the other end of an IRC client or mailing list.

Crucially, they recommended a bar serving excellent beer.

Posted in Uncategorized | Leave a comment

Leave a Reply