The Advent Awakens

One year after the destruction of the eggnog, a deep chill once again grips the republic.  Frost glitters in the hearts of those who would seek to rule, and the light sabres of the Jedi remain relentlessly dark.  A new resistance is needed, to lead the fight against short days and long, dark nights.

And that is where we, the combined forces of IAM and UPS come in, bringing hope to the galaxy, one door at a time.

1st December (Dave)

GhostThe Ghost drifts silently through  the shadow of a moon, hoping to avoid detection by the enemy, as the rebels gather intelligence ahead of making their first move.

2nd December (Alex)

Sabine Wren stands in front of the Ghost, blaster pistol in her left hand, facing outwards.The Ghost gently touches down on the edge of the icy wilderness and Sabine Wren steps out, quickly moving so that the ship is very far away just behind her so as not to break scale. She quietly and confidently rocks her hairstyle while she waits for the Phantom to arrive.

3rd December (Jim)

As the Phantom settles gently onto the landing pad accompanied by the familiar hiss of its gas evacuators, the rebel pilot turns to his companion and smiles expansively: “I really love the new Mk III Phantom!

The co-pilot’s head swivels sharply: “No. This is the Mk II. The Mk III has the chromite turbine fan inlets, and those as you can clearly see are vanadium!

The pilot’s face colours: “You’re thinking of the limited edition Mk IIa with the carbonite dorsal injector couplers, and anyway the Mk II was subject to export embargoes and was never available for sale in the Galactic rim!

The co-pilot refuses to yield: “You’re mistaken! The Mk II *was* subject to export embargoes but a small number were sold under license by Aratech Technologies inside the Minos Cluster! Also the Mk III never included the rear shield compensators which, as you can plainly see, are shown on the dashboard readout!

Desperate to resolve the argument, they simultaneously reach for their datapads – just as the Imperial assassin droid’s missile impacts and reduces both men and their Mk IIb Phantom (neither had noticed the crynex-shielded torque vent beneath the weapon housing or the optional pistol rack mounted behind their head-rests) to their constituent atoms.

4th December (Chris T)

As the blaster cannon plummets to the ground, curiously self-aware, it channeled a bowl of petunias, thinking ‘Oh no, not again!’. Exactly why this thought went through its head is unknown, but it has been speculated that if it was understood the universe would implode.
Upon impact, the cannon fired unexpectedly, injuring Sabine and causing her to dive to the ground for cover.

6th December (Gary)

I told the storm trooper it was my ship.

7th December (Dave Shipway)

OK lads, very funny,” growled Stormtrooper THX1139.  “Who has been using my helmet as a potty again!?  If I find out who did it … well, I would threaten to shoot them, but you know I always manage to miss anything I’m trying to hit … but they’ll be getting my boot up their thermal exhaust port I can tell you!

“I don’t know, first of all I get stuck on the garbage Compactor maintenance rota all week, and now this.  It’s enough to make me wish I had joined the Rebel Alliance … at least they dont’ have to wear this stupid armour that doesn’t even protect you from an ewok with a catapult!”

8th December (Adrian)

Nobody ever gives us weapons-racks any credit in these sagas. And the cheek of it all, having to carry a wheel-spanner for that stupid mark two thingy in case it gets a flat!

9th December (David R.)

Just look at this beauty.  Rey’s speeder has landed.  Rey has gone down to Diogenes’ Barrel for a hair of the cynic.

10th December (Fiona)

Unkar’s Thug dusted the snow off his lapel as he climbed the peak of The Ice Mountain.  He grimaced, realising that he may never see Niima Outpost again.

11th December (Dameon)

Rampant LuggabeastOverexcited as usual, the friendly local luggabeast mistakes Taslin Brace for the postman knocking him clean over in the hunt for presents being delivered… No luck today, but just a few more sleeps to go!

12th December (Jim H)

The conditions were treacherously icy, and Han wasn’t looking forward to a hill start in the Falcon. Admiral Akbar pointed out that they were expected to make every reasonable effort to attend for work as normal, so Han begrudgingly fetched his ice gear while muttering about Tauntauns.

13th December (Jim L)

Climbing up the side of the valley, Stormtrooper THX1139 looks back down at the platoon’s First Order Snowspeeder and is surprised how pleasantly the strong lines of the machine contrast with the softly undulating snowmounds sleeping about its base.

He trudges upward, the patient crunch of snow under his boots echoing like the gentle tear of wrapping paper in the hands of an expectant child. Puffs of ethereal mist momentarily envelop his helmet as he steadily breathes in and out. But he journeys on, resolute, and strangely at peace in the revolution of snow.

Glistering in impeachable beauty the sunlight reflects crazily off surrounding ice crystals, beams of light cast from their prismic edges dancing joyfully across his visor. And as he journeys upward, an unfamiliar feeling starts in his breast.


As he tops the rise and looks down on the approaching rebel convoy, a gentle smile tugs at the corners of his mouth.

“I bloody love ambushes!”

14th December (Jok)

Trevor and Barry from accounts were well known for their tank tops, Casio calculator watches and black shoes with white sock combos.

It had been a strange office Xmas party indeed.    Their annual attempt towards being humorous attaching Mistletoe to their trousers had once again left them shunned into the corner.
“Who would leave a dentist’s chair here?”, enquired Barry.   Easing himself into the chair they were approached by a mystical stranger from a far off land.    The stranger, with a curious Glaswegian accent, produced a bottle.   It was like no other imbibed in their presence. “So, ya’all be huvin a wee snifter at ra’ expense o’ Mr Jäger wull ya noo”, said the stranger sporting a wry smile.  After being encouraged to shoot 6 straight Jägerbomb’s in the dentist chair Barry and Trevor strutted like Bee Gees onto the dance floor.   With moves that Justin Timberlake would later emulate reviving his flagging career they danced.  And oh how they danced.   Girls swooned as their plastic patent shoes reflected stars from the rotating ball, which only that night been purchased for £10.99 at Cowley Road Liddle’s.   Barry was now being chanted with furore to dance faster and faster.   “Baz, Baz, Baaaaazzzza”, the workplace chanted and in unison Trevor became cool and would be forever known as Nevil.   And with dance, and music and another half dozen shots of Jäger the night went on and on.  Another 5 shots and the stranger said, “Come, it’s time”.   But to where they both looked inquiringly.   And as the ravers revelled on like Pagans before a feast, they stepped back to the chair.  “Sit”, the stranger motioned and Nevil did as he was told.   “Not on the floor you great wazzock, get in the chair”, said the stranger with a sighing breath.    And Nevil sat in the chair, and all was comfy and good.   “Do you think I could be excused to the loo?”, asked Baz.   The eyes of the stranger rolled and said “If you must”.   The Jäger was taking Baz to new levels, he was now sailing to Neptune on a magic swan as the music changed from Kylie Minogue to German techno Neo-thrash daftpunk music seeped into every pore of his being.   Hours later he came out the toilet.   His head now shaven into a Mohican, eyebrows removed and profanities written onto his forehead in lipstick he felt ready for anything.   The stranger was indeed pleased.
And Baz joined Nevil in the chair, another 6 shots the stranger started spinning the chair.  Faster and faster, round and round it spun.   The Pagans arrived from the dance floor and with each revolution their voices soared. Then the strangers hand brought the chair to a sudden stop.   Their heads spinning they stumbled onto the dance floor. 
They had no idea where they were.   A red spotlight shone from behind Baz.    “I think, well I think that I think that I’m thinking we are on another planet”, said Nevil.   Baz stared and Nevil and Nevil in turn back at Baz.   “Do you know you are carrying a weapon”, said Baz.   “Oh, say’s you”, said Nevil.   “And you think I haven’t noticed that you have changed out of the tank top I bought you for last Christmas’s and now you’re wearing some weird power rangers motorbike helmet thing”, he continued.   Baz couldn’t contain his confusion any longer.  He started making R2D2 noises.  “Why are you doing that?”, enquired Nevil.   “I think you’re supposed to be a Stormtrooper”, Nevil said knowingly.   “Oh, sorry I only know what noise an ewok makes”, and began singing “me tabby yab yab”.   Nevil responded despairingly, “I don’t know what noise a storm trooper makes but it’s not sodden me tabby yab yab”.
And then a voice boomed from the heavens….. “Yer a Snowtrooper”.   Where it came from neither knew.   “Er, is that your boss on the intercom”, said Baz sheepishly to Nevil.   
“I’ve seen you squeezing all the secret Santa gifts”, said the voice.   “Did you really feel his presents?” asked Baz.   Just then, a former Oracle DBA contractor came into the room.   “Des!, why are you here?”, they both exclaimed.   And with that Des, drunker than a Cornish pirate on a stag do, fell onto the floor, his bottle of Jägermeister smashed into a thousand bits.
The music stopped, the lights came on.  Everyone was staring at Baz, Nevil and Des.  Then the voice from the heavens boomed again “Pick him up you drunken fools, I think he has broken his legs”.   And Des held aloft his arms reminiscent of Sergeant Elias from Platoon.  His eyes interlocking with theirs. And Baz looked at Nevil knowingly and said……. (wait for it….!)
“I think it’s time we embraced our Des two knees”
15th December (Robert)
Snow machine pushed by a storm trooper with Millennium Falcon in background
“Let it snow, let it snow, let it snow…” sang the stormtrooper as he pushed the snow machine, lightly dusting the Millennium Falcon as he went.

17th December (Mark)

The silver surfer looks down on Galactus’ Christmas bonbon and thinks to himself … is this the wrong franchise?

18th December (Michael)

“They sure are prettier down there,” the Y-wing pilot thought to herself as she admired the Christmas lights on the wreckage of the Imperial Star Destroyers. “They’re no fun to meet in battle – but they are a nice background to the Christmas trees!” Fortunate, too, as they were the landmark that showed they were nearly back at base – complete with the Christmas pudding supply.



19th December (Nigel)

Feeling a bit fuzzy at the edges from the stray blaster cannon shot, Sabine gazes out onto the icy tundra, daydreaming about the coming Christmas family reunion, completely unaware that cousin Kylo has already arrived, as he quietly lands his TIE Silencer onto the ice flow in the near distance behind her.






20th December (Jim H)

The tank driver was glad to have put enough distance between himself and the Cantina that he could no longer hear the band’s one song, but he blasted it to rubble just to be sure.





23rd December (Steve)

The newly created dark side force ghost mounts the battle ready Christmas
attack sleigh intent on intercepting Santa.
Can Santa survive the imminent encounter and save Christmas ….?


24th December (Dave)


There’s always one who takes things just a bit too far.  The Imperial Walker will not be amused.

Happy Christmas all!

Posted in Star Wars Advent | Tagged | Leave a comment

FLOSS 2017

I usually spend most of my time safely hidden away in an attic full of geeks, but the evening of March 14th saw me embark on quite a voyage of discovery. I was on a train heading way up north to Manchester (I’m from the Isle of Wight originally, so anything north of Cowes is ‘north’ as far as I can tell, hence my apparent difficulty with geography).

The department had suggested I would enjoy a visit to ‘FLOSS Spring 2017’, the annual meeting of FLOSSUK, formerly the UK Unix User Group where I would have the opportunity to meet like-minded Linux users and get to hear about developments in the open-source world from a non-corporate viewpoint.

Things got off to a promising start – as the outskirts of Manchester slid past the train window I was encouraged by the lack of riot police and burning tyres that I had been expecting. Actually, it turns out that Manchester is a great venue for this type of event. The various hotels, transport links and ‘The Studio’, the chosen location for the conference itself are all located within easy walking distance of each other in a city centre that seemed more vibrant and accessible that some of the locations I have visited in London for these sort of events.

The conference itself turned out to be smaller in scale than many of the ‘corporate’ events I have attended in the past, but what the assembled crowd lacked in numbers they more than made up for in their enthusiasm at being part of FLOSS.

Having checked for the essentials (i.e. a more than adequate supply of caffeine and tasty snacks) we gathered for the opening keynote speech by VM ‘Vicky’ Brasseur, an IT professional and public speaker who describes herself as ‘an advocate of and evangelist for freedom and openness in all things’ and we learned of her bold vision of being able to use open source tools in a corporate environment without losing the ‘open-ness’ of that approach.

There followed two busy days of presentations on a wide range of topics ranging from individual life-at-the-coalface experiences with open source tools to much more technical discussions involving, for example, such topics as OpenSSH certificate management techniques.

It turns out that the FLOSS set are a tolerant bunch of people – we allowed Gavin Atkinson, who is both one of the FLOSS committee and a developer for FreeBSD to evangelise about… um… that other Unix-like operating system (you know, the one with the little red devil logo) and he didn’t have to dodge even one egg or rotten tomato! This feat of self-preservation on Gavin’s part allowed him to make an interesting point – we as a community spending less time worrying about which flavour of Unix to use and more time on getting stuff to work and then getting it ‘out there’.

Some other high points of the event were the ceremonial unboxing of the very first KDE SlimBook fresh off the production line. There were also the regular ‘Lightning Talks’ where each speaker gets exactly five minutes to deliver a presentation on their chosen topic (my favourite being – Quake 3 on a Raspberry Pi. What’s not to like?)

Overall thoughts? Well, I went from almost having to be physically dragged to the railway station and thrown on a train, to returning full of enthusiasm for the event and the community that organises it. The venue chosen was excellent and the variety of speakers and the topics they presented meant there was often a difficult choice to be made about which talks to attend as they all sounded so interesting. Would I go again? Definitely! (hint… hint…)

Posted in Conferences | Leave a comment

Kerberos upgrades: rekeying the krbtgt

Kerberos is the University’s Single Sign On system, which underpins other services such as WebAuth and Shibboleth.  Most members of the University don’t use it directly, but indirectly use it every day.

After something of a delay, we are continuing with our Kerberos upgrades, as previously described.

Having successfully upgraded kdc-admin, it’s on to the krbtgt/OX.AC.UK principal – that is, step 2 (and then step 3) of the “What will this work involve” section.

While we are announcing the work to IT Support Staff (ITSS) in the University, this blog post is to provide more background, and explain why we’ve made some of the decisions we have.

What is the krbtgt?

When you successfully authenticate to a KDC, you are given a TGT (Ticket Granting Ticket).  This is passed back to the KDC when you want a ticket for another service, and proves that you are who you say you are.  This ticket is encrypted with the krbtgt principal for a realm – so in our case, it’s the krbtgt/OX.AC.UK@OX.AC.UK ticket.

The only systems that know the password for the krbtgt are the KDCs.

What are we doing?

At the moment, the krbtgt/OX.AC.UK principal only supports DES and 3DES encryption types.  DES has been deprecated for years, and as of 2015, MIT (who develop the version of Kerberos we use) have removed it from the default supported encryption type list.

We are going to rekey the krbtgt to add RC4 and AES encryption types.

We will continue to support DES and 3DES on the principal until we have established that no-one is using it.

Due to an interesting quirk, our krbtgt actually has two DES keys: des-cbc-md5 and des-cbc-crc.  When we rekey, we will drop desc-cbc-md5, as it is not possible to add multiple keys with the same encryption type.  We have established (via the logs on the KDCs) that no-one is currently using des-cbc-md5.

Our plan is pretty much the MIT DES retirement plan.  We will keep the old keys, so existing sessions continue to work.

Why has it taken so long?

Back when we tried this in 2015, we discovered an oddity while doing some final testing.  As we can’t easily roll back once we’ve gone live, and we didn’t understand what was happening, we decided to roll back and investigate further.

It took a while, but we tracked down the issue.  If you get a ticket before rekeying, rekey, then forward your ticket (eg via SSH) and try and use it, you get a “bad encryption type” error.  (There is more detail in the mailing list post I wrote about it.)

The MIT Kerberos developers replied to say that this was a new manifestation of a known bug, that was fixed in Kerberos 1.14.  (It has since been confirmed that the same thing will happen if you have a renewable ticket, and try and renew it rather than getting an entirely new ticket.)

(Just to note here, a renewable ticket is a specific type of ticket that you can present back to the KDC to extend its lifetime.  Normally, you would have to re-authenticate (with a password or keytab), and get a new ticket (which is the behaviour of tools like k5start).  If you use krenew, this will affect you.)

The problem here is that we are using Kerberos 1.12, which is the version currently in Debian stable (jessie), and upstream suggested that it would be difficult to backport the patch.  That’s not too much of an issue, though – Debian testing (stretch) is close enough to release that we can backport the 1.14 libraries from it, and use them.

We did this, and in November we rolled out some new KDCs running Kerberos 1.14 (these replaced KDCs running the very old 1.8).

Unfortunately, by the end of the day we had 4 or 5 reports from ITSS with cross-realm trusts to their Windows domains that users could no longer access file stores when using the new KDCs.  As we had only upgraded some KDCs, they were able to test against the new and old KDCs, and identified that the problem was with the new KDCs.

After some head-scratching, we rolled the new KDCs back to 1.12, and suddenly all the problems went away.

With the generous assistance of Simon Wedge at St Antony’s, who had a system that was consistently failing, we got some packet dumps, and were able to analyse them.

It seems that between 1.12 and 1.14 MIT Kerberos changed the way it responded (also) to initial authentication requests.  In 1.12 and earlier, it would return a list of all encryption types supported by the principal for which authentication was being tried (which included DES and 3DES).  However, for 1.14, it only responds with a single encryption type (generally the strongest – which is not DES or 3DES!).

Windows was somehow caching this result – but not using it initially.  Instead, it would use the full list of encryption types for the complete initial authentication, and get a valid ticket.  It would then attempt to re-authenticate to access a file share.  At this point, instead of sending the normal list of encryption types, it would send the one that was returned by the KDC earlier, and its own list – which included RC4 and some custom Microsoft RC4 encryption types, but not DES.  This then failed, because the krbtgt didn’t have any keys of that type.

Rather confusingly, we only saw errors from some cross-realm trusts – we know of at least 3 or 4 other Windows cross-realm trusts that worked fine.

Now, there is a work-around suggested by the developers – unfortunately, this effectively makes all krbtgt tickets DES, even those that could be 3DES.  This is something that we are keen to avoid.

Ironically, once we rekey the krbtgt, the 1.14 problem goes away, as we will have the full set of encryption types supported by the krbtgt.

Rock and a hard place?

So, we find ourselves with a choice – stick with 1.12 (which we know has issues with renewable tickets) or upgrade to 1.14 (which will break cross-realm trusts for a time).

The risk of upgrading to 1.14 is that if things break we can’t necessarily easily tell whether it’s caused by the rekey or 1.14.  With 1.12 we have been running it for over a month, and have a good feeling for what is ‘normal’.

1.12 is also the version currently in Debian stable – 1.14 would require us to track Debian testing and backport any appropriate fixes (made more interesting by the fact that since we started this, 1.15 has moved into testing – so we’d have to backport and test that).

We have therefore decided that we will stick with 1.12, and accept the risk of renewable/forwardable tickets not working.

When exactly is this happening?

Our standard maintenance period is 7am-9am on a Tuesday morning.  This is partly because it coincides with the Janet maintenance period, and partly because if anything goes wrong staff are available during the day to fix problems.

We expect any issues to fall in to one of two groups:

  • ‘transient’ issues with sessions, where sessions created before the rekey do not work post-rekey
  • ‘permanent’ issues, where systems do not work with the rekeyed krbtgt

The default Kerberos ticket lifetime is 10 hours, so the permanent issues (with new sessions) may well only become apparent some time after we make the change – if we’re unlucky, about the time everyone is going home.

For this reason, we have decided to make the change on at 9pm on Monday evening.  This should minimize the number of people who see issues with existing sessions, purely because there are fewer people using the system at night out of term.  It also means that if permanent issues appear we can work with ITSS colleagues to identify and fix them during normal working hours.  (It also means that we shouldn’t end up working a 16-hour day – towards the ends of those, troubleshooting gets very hard.)

We are doing this on Monday 9th January 2017, which is Monday of 0th week.  This is less notice than we would ideally like, but this date is a compromise with the minimal number of users actively using systems.

What impact will I see?

Hopefully, none.

We have tested that Webauth works fine (unless you’ve done something very non-standard to your server).  Shibboleth will also work.  So, most people shouldn’t notice.

We have tested cross-realm trusts (a simple case with Server 2008R2 and Windows 7, and Server 2012 and Windows 10), and they work in testing.  However, given the different setups across the University, this is in no way a comprehensive test (as we saw from the 1.14 upgrade – a handful of units had issues where most were fine).

What if I do see problems?

If you are an end user, please talk to your local IT Support Staff.  They will be able to assist you in identifying the issue, and should be able to assist you with initial investigations.

If you run a service that is affected, we recommend you restart the affected service, or, at the worst case, reboot the systems.  While this sounds a very stereotypical answer, it is for good reason – it will clear any state may have been using the old encryption types, and also fix any renewable tickets, if they existed.

If that doesn’t work, please email us at giving as much detail as possible.  We will be able to review the logs on our side, and help troubleshoot and fix your problems.

What if it all goes wrong?

If everything goes pear-shaped, we will be able to roll back.

Unfortunately, this will invalidate the sessions of everyone who has got a new ticket since the rekey.

This could potentially have a large impact – while WebAuth should be ok (people will be asked to re-authenticate), other services will likely experience issues until they get new tickets.  This includes many IT-Services run systems (including anything backed by Oak LDAP, the Registration service, the mirror service, mailing lists, CUD).

It is possible that the rollback may also roll back all changes since the rekey – including account creation and deletion, and password changes.

The impact of this is likely to be so large that we would prefer to work with ITSS to fix problems, rather than roll back and then deal with restarting services.

What’s next?

If this works, we expect most principals to move to using AES256 tickets immediately.  Once things have settled down we will follow up with owners of principals that are still using DES, and help them move to a stronger encryption type.

Once we have no users of DES, we will be able to rekey again (which will be much less painful, as we’re not changing the strongest enctype) and remove DES entirely.

Posted in Service Improvement | Tagged , | Leave a comment

SysDev Star Wars: Attack of the Advent

In a galaxy quite close to home…

Regular readers should be encouraged to discover that our adherence to this great tradition has not wavered, as we not only enjoy some advent fun in the run up to the Christmas holidays, but also remember a good friend and colleague, despite being gripped by the closing in of the dark side – until Spring at least.

The rules remain the same, each day one door will be opened, and one brave Jedi will do battle with fiddly little pieces of plastic, and then triumphantly append a photo of their finished masterpiece to this post.

We’ll be nagging team members to update this blog post daily with their latest lego adventures (with possible delays over the weekends). The first update should appear very shortly…

1st December 2016 (Adrian)


Many many aeons after Jango Fett had stolen the patrol craft and taken it as his own and named it Slave, a strange event occurred. A new venture to explore the outer reaches of the galaxy was about to commence. Then, out of the early frosty and wintry morning a new ship appeared. But not one of these new-fangled jobs. It was an absolute clone of the Slave. Where it came from, no-one knew. But it was seen hovering in front of a set of old blue-prints of the Slave, and the myth arose that it had spontaneously auto-reassembled from thin air. It henceforth became known as henceforth-the-auto-reassembled.

2nd December 2016 (Alex)


This poor Bespin guard bursts from his plastic and cardboard prison, takes a moment to compose himself — luckily he is of few parts — and looks around. This was not Bespin, as evidenced by there being a ground to stand upon. Turning, he discovers an abandoned patrol craft and wonders whether he might fit inside. Thinking it unlikely and unsure of what else to do, he resolves that he should do what he knows best. He stands and guards it, waiting.

3rd December 2016 (Christopher H)

20161209_140508Fly, my pretty! Fly!

4th December 2016 (ChrisF)

A passing Imperial Navy Trooper swaggers on to the scene and sees a bored looking Bespin Guard who seems to have developed a fondness for a piece of nearby space junk. Naturally the Bespin Guard takes offence at this description of the recently discovered patrol craft. Honour must be satisfied!

So, the duel begins… Ten paces, turn and vapourise! But who will be the victor?

5th December 2016 (Chux)

A DF.9 gun turret guarding a precious christmas tree

Trying to make the most of the temporary refuge which Echo Base had become, General Rieekan orders for the installation of DF.9 gun turrets at vantage points around the base.

This particular DF.9 was hastily installed to contain a clutch of Imperial AT-AT walkers and ground troops spotted making a bee-line for a particularly precious Christmas tree.

And just in time too, the gun turret helps defeat the Imperial forces and saves Christmas at Echo Base!!!

6th December (Dameon)

2016-12-06_snow_trooper_smAnd now there are three of them, it turns from a casual duel, to an official Mexican standoff!

“Get off my snowfield, Bespin scum!” yells the Snow Trooper, “I don’t even know what a Mexican is!”

“I don’t think we have them in this far far away galaxy” replies the Navy Trooper, before turning his blaster on the cloud-city native, all the while keeping one eye on the distant gun turret.

7th December (Jim)

2016-12-06_snow_trooper_smOlaf the Snowtrooper Snowman has just come down the mountain.

But where are Anna and Kristoff? We were due to go up the mountain to find Elsa!

And which bastard took my stick arms?

He then looks to his left and sees the HX400-UT Imperial Blaster Cannon and ejaculates:

Bollocks! I’m in the wrong movie!

8th December (Dave)

20161208 A resupply mission gives the Mexican standoff a turn to the unfair when the Imperial Navy Trooper finds himself armed with a dish canon.

“Feel the POWER of MY Force! Yeah, baby! This is what I’M TALKIN’ ABOUT!”

“Will? Will Smith? Is that you?” asks the Bespin Guard.

“CUT CUT CUT CUT CUT! screams the director. “Will you guys PLEASE stick to the script? You ain’t the Fresh Prince of nothin’ out here and your ad libs ARE NOT better than the lines we gave ya. Okay, ready people? Let’s make some MAGIC today! From the top, ACTION!”

9th December (DR)

goggsWhere the hell did I put my goggles?

10th December (Jim)

Venator-class Star DestroyerNewly assigned to his post as Admiral of “Constipation”, the Galactic Republic’s new Venator-class Star Destroyer, Lar Jarse, stood on the bridge and surveyed the life teeming on the planet below.

Well, we can’t have this! All these creepy-crawly things squirming around to no good! It just isn’t British!

As his crew desperately scrambled for their scanners, the new Admiral vociferated and the unfortunate planet’s fate was sealed.

Unleash the death-ray!

11th December (Julian)


Back on the other nearby planet an Armoured Assault Tank swings into view. Its clone pilots are briefly confused not to see the Gungans that they were originally pursuing. Ah, well lets take out that Bespin Guard first and then see whether we get any further orders from our leader – hang-on maybe that carrot nosed trooper is our leader?

12th December (Ken)


“Mayday! Mayday! Mayday!


13th December (Robert)

Battle droid “Hold it right there!” yelled the battle droid, as it came across Olaf.  “Give me the battle cannon or else!”

Olaf sighed, and continued to stare into the distance as he wondered just where he had ended up.  “Just let it go…” he muttered…

14th December (Michael)


As Obi-Wan takes a  sharp right around the festively-decorated turret, he idly wonders how he’s being persued by a ship that wouldn’t be invented for many years.  The magic of Christmas?

15th December (Dameon)

2016-12-15_tantive_iv_sm“What do you mean ‘The damage doesn’t look so bad from out there’?” exclaims the increasingly pessimistic Captain Antilles.
“‘Oh, there’s the Tantive IV’ they say, ‘Let’s pull them over and check their systems for secret plans again. That’s always a good laugh’ … damned imperial speed cops, don’t they have anything better to do with their time?”
“Oh well, brace for impact … again…”


16th December (Robert)

E-3P0Meanwhile, E-3P0 was admiring the Empire’s latest Star Destroyer, “Constipation”.  Whilst he was well aware of its ability to destroy entire planets with a single shot, from here it looked small; almost insignificant compared to some of the other ships he had seen lately.  Such a powerful ship deserved a name with distinction, a name with gravitas.  “Constipation”? That has zero gravitas, he thought.

17th December (Stu)


A GNK Power Droid arrives to charge up the parked Starship Enterprise*. “It’s a lot smaller than it looks on the telly.”, it gonks.

[* Pesky wormholes, mixing up the universes.]




18th December (Adrian)

20161220_jabba_palace-1Of course, all along Jabba (the Hutt) had been up in his palace atop Mt Pannatooine, observing  the arrival of lots of goodies for the plunder down in the valley below. He was licking his slobbery chops and anticipating a good festive season. What he didn’t know though, was that his musings were to be rudely interrupted by a swarm of raisin-bots which had scaled the steep cliffs and were intent on some plundering of their own.

 19th December (Alex)

Luke who it is! The Death Star Trooper tries to take aim but finds himself lifted into the air as the protocol droid looks on. Better style this one out, he thinks. “Thanks, Matilda!” says Luke to the small girl hiding in the Gonk droid.




20th December (ChrisF)


Fast forward… and our once heroic Imperial Navy Trooper, having eradicated every last trace of Rebel nonsense in the neighbourhood (especially that Bespin rogue with poor dress sense) is now bored witless and is reduced to handing out on the spot fines to illegally parked Desert Skiffs for a living. It was either that or a job grilling rontoburgers at the local “StarChow! All the nutrients you can chew, suck or absorb for only 99 Imperial Credits!”

21st December (Christopher H)

Lounging on his Sun-bed, the tired soldier saw an approaching airborne object. “Is it a bird? Is it a plane? I’ll shoot it down anyway he thought”.

22nd December (Stu)


An Imperial Sentinel Class Landing Craft cruises around with Luke roof-surfing, before doing what it does best, and landing. “Look at me! Olaf. Look at me! Olaf! OLAF! YOU’RE NOT LOOKING!!”






23rd December (Nigel)

Having not only gained Olaf’s attention, but also an impromptu lecture about how “Star Wars” is “so last millennium, baby” and the future is in Ice and crossover movies, “like, um, ‘Star Wars Frozen'”, Luke trades his father’s lightsaber for a pair of ice skates and a hockey stick, determined to hone his Jedi Hockey skills (puck telekinesis, anyone?) for the musical “Disney On Ice: Star Wars” that Olaf makes him certain is just around the corner.

Feel the freeze, Luke, feel the freeze.


24th December (Dave H)

snow-chewbaccaThe Albino Chewbacca is decked with festive decoration – his bandolier is painted red and green. He comes with a snazzy new bowcaster which fires off snowballs (1×1 studs) which is always a great weapon to have. He also comes with two miniature pine trees and some spare snowballs.

A great way to finish the 2016 Star Wars Advent calendar.

Happy Christmas!

Posted in Star Wars Advent | Tagged | Leave a comment

Shibboleth Identity Provider upgrades

After some slight prompting by both the Networks team and colleagues in Sysdev, the IAM team felt that we should write some blog posts of our own about our own work to upgrade the University’s authentication infrastructure.  The first of these is on our work to upgrade the Shibboleth service.  This work ensures that we are running a fully-supported version of Shibboleth, as well as enabling new features in the future, such as single log-out.  The upgrade will also make our Shibboleth servers highly available, which should improve service reliability, and allow us to consolidate our existing servers to an extent.

The upgraded service will go live on the 5th April after much testing over the past few months.  No Shibboleth-protected services should be affected by this work, and the upgrade should be transparent to end users.

What is Shibboleth?

Shibboleth currently sits at the top of Oxford’s Single Sign-On (SSO) stack, on top of both Kerberos and Webauth.  The original purpose of Shibboleth was to extend SSO to services outside the University, such as journal access.  However, Shibboleth is also frequently used for services within the University as well, not least to provide SSO to systems that lack support for Webauth.  Although Windows servers are the most common case of servers without Webauth support, other systems such as Bradford Campus Manager also fall within this group.  Shibboleth is based on the idea of “claims-based authentication” using SAML, where a Service Provider (or SP) is given a signed “assertion” from a trusted Identity Provider (IdP).  This assertion contains details (known as attributes) about the end-user such as username, name and email address that can then be used to make decisions about access.

For Shibboleth to work, the IdP and SP need to know certain details about each other, such as where they may be found and the certificates used for signing assertions.  This information is known as the metadata for a given server.  It is possible to share this manually between the two servers if needed, but when this is scaled up to a large number of services and identity providers it becomes unwieldy to manage the metadata swapping.  To solve this problem, Shibboleth servers generally have their metadata published by one or more “federations”, which act as a single trusted source of metadata.  The individual Shibboleth servers then fetch signed metadata from the federations they trust.

Since Shibboleth may be used to login to many different SPs with varying levels of trust, the software is privacy-preserving by default.  This means that attributes that could be used to identify end users must be explicitly “released” to a given service provider.  This means that instead of a normal username, services are typically presented with an opaque persistent ID, which is generated by a one-way hash of the service provider’s “entityID” (an identifier for that particular identity or service provider) and the Oxford SSO username.  This prevents separate SPs working together to de-anonymise users.

Why upgrade Shibboleth?

About a year ago, we received the news that updates and support for version 2 of the Shibboleth Identity Provider (IdP) server would be discontinued by July 2016.  This meant that we had to start work on migrating to the new version of the software (IdP v3), since running supported software is a good idea.

In addition to the obvious desire to run a supported version of the IdP software, the upgrade also means we can make resiliency improvements.  At present, almost all Oxford Shibboleth authentication is handled by a single server.  This is mostly down to the difficulties in setting up an IdP v2 cluster, but is also down to avoidance of load-balancers in the past.  (For historical reasons, there is also a completely separate IdP pair that is used for some internal business systems, with manual switching between the two servers.)  However, the popularity of Shibboleth for new services means that the current single point of failure is no longer a sensible option today.  The IdP v3  software is also rather easier to cluster than the previous version, and no longer requires a complicated state-sharing mechanism for clustering.

Finally, the upgrade process provides an opportunity to consolidate our existing Shibboleth environments.  Currently, we have three environments, which look like the following:

  • Main IdP
    • Live (1 server)
    • Test IdP (1 server)
    • Development (1 server)
  • Business Systems IdP
    • Live (2 servers)
    • Test (1 server)
  • IAM test stack (1 server)

As mentioned earlier, we have historically run a separate IdP for business systems that required a high-availability authentication service.  However, as the upgrade will bring high-availability features to the main IdP, we should be able to remove the additional environment:

  • Main IdP
    • Live (3 servers)
    • Test IdP (2 servers)
    • Development (1 server)
  • IAM test stack (1 server)

While the total number of servers is identical, the elimination of the two business systems environments improves manageability of the service.

Load balancing and improving resiliency

The new service uses the Netscaler load-balancing device run by the Business Systems Operations Team, which is also used by WebLearn and other services.  The Netscaler supports both session stickiness (necessary for avoiding server switches mid-authentication) and content-based switching, which is useful for allowing users to choose between old and new servers as well as separating out SAML1 and SAML2 requests for testing.  For services using SAML2, the attributes are transferred between the IdP and SP via the end-user’s browser.  However, in the case of SPs using SAML1, the SP must contact the IdP directly via a back-channel to obtain attributes.  All the necessary state is stored on the client side, so no shared server state is required.  The only exception to this is the authentication process, which must be performed on a single server.

One interesting question is how the IdP maps an attribute query to the back-channel to SAML1 authentication request to the front-channel.  The answer is that the front-channel returns a transient ID which is reversibly encrypted.  The back-channel process then decrypts this transient ID to find out which user the request applies to.

Problems we saw

While the process of upgrading was slow, there were relatively few problems during the upgrade process.  In several cases, the upgrade to IdP v3 improved compatibility with external services.  For example, some service providers require particular types of authentication or require certain forms of user identifier to define the “subject” of an assertion.  However, there were some problems that we saw during the upgrade.


The first problem was how to test service providers that still use the old SAML1 protocol.  Because these servers communicate directly with the IdP to retrieve attributes, it is generally difficult to test whether these behave as intended with the new service.  The solution we came up with was to test specific development servers against the new IdP cluster, before testing external systems later in the rollout process.  Ideally, we would have tested external sites with a separate test IdP.  Unfortunately, some providers set strict limits on the number of IdPs that can be trusted (often 1) for a given organization, which makes this impossible.

Assertion signature algorithm

Another problem we saw was a lack of support for assertion signatures based on SHA-2.  This is fairly rare, but affected one relatively important Service Provider: the cloud-based software used by our centralised helpdesk.  While some may consider a lack of visible queries to answer a good thing at times, the Service Desk team may beg to differ!  We fixed this by modifying relying-party.xml, as documented in the Shibboleth wiki:

<!-- SHA-1 support bean -->
<bean id="SHA1SecurityConfig" parent="shibboleth.DefaultSecurityConfiguration"
  p:signatureSigningConfiguration-ref="shibboleth.SigningConfiguration.SHA1" />

<util:list id="shibboleth.RelyingPartyOverrides">
  <bean parent="RelyingPartyByName" c:relyingPartyIds="entityID here">
    <property name="profileConfigurations">
        <bean parent="SAML2.SSO" p:securityConfiguration-ref="SHA1SecurityConfig" />

Persistent IDs

The third issue we saw concerned our generation of opaque persistent IDs, which include an IdP-specific salt value.  This is needed so that SPs cannot trivially reverse the persistent ID by brute force.  For historical reasons, we use a random binary salt as opposed to the text-based salt more typically used, and accommodating this required some minor modifications to the IdP software.

Additional Verification

The final problem we saw was with the Additional Verification service, which provides multi-factor authentication.  Although this service is rather limited at present, Additional Verification is currently used by WebLearn to protect examination setting and marking.  The service is currently based on a custom-written Java servlet that sends one-time codes via text message.  As the new IdP version changed the authentication interfaces used, the servlet required some modifications to work correctly.  As a side-effect, the service was also restyled to match the current Webauth service.

The roll-out process

We started the process on the 8th March by placing our existing IdP behind the Netscaler load balancer.  The existing server kept its IP address, but the DNS entries were modified to point at the load balancer.  The reason we did this was to avoid problems with SPs that use the older SAML1 protocol, which include several journals and library resources, along with the Bodleian’s SOLO portal and this blog.  Since some SPs cache DNS responses for up to seven days, a grace period is needed to make sure that the back-channel and front-channel connections both use the load balancer.

Netscaler setup before IdP v3 go-live

Netscaler setup before IdP v3 go-live (courtesy of Julian)

The next step was to test that services using the old SAML1 protocol still worked using the new servers.  On the 22nd March, we temporarily switched requests for SAML1 authentication (including back-channel requests) to the new servers during the maintenance window.  This let us test that the new servers worked as intended with external journals, and confirmed that the sites worked.

The final step will be to switch traffic from the old IdP server to the new cluster.  Barring any last-minute problems, this will happen on the 5th April during the 7 a.m.-9 a.m. maintenance window, which will allow us time to test the new service and revert back if anything does go wrong. The resulting Netscaler setup will look like this:

Netscaler setup after IdP v3 go-live

Netscaler setup after IdP v3 go-live (courtesy of Julian)

Posted in Service Improvement | Tagged , , | Leave a comment