Return of the Advent 2

December 1st

lego spacecraftThe Mandalorian touched down on the head of the sleeping womp rat, hoping that his beskar armour would be a match for it. Kuiil had warned him, “Five ends of a womp rat are sharp. I have spoken.”

December 2nd

“I am going to stomp so much mud into your laptop for this” plotted the womp rat, unamused by Riot Mar’s plastic starfighter, or that hydrocarbons were frittered on purposes other than womp rat warming.

December 3rd

lego stormtrooper by webcam“Teams xmas party!” enthused the stormtrooper, imagining all the cocktail stick snacking opportunities. “Timesheet it under ITS00A, xmas party replacement project, and don’t invite the cabinet secretary”. Raising a skewered cube of non-dairy alternative he declared, “That’s no moon”.

December 4th

lego stormtrooper holding heavy repeating blasterThe stormtrooper guarded the womp rat carefully, as the Index transited to some dangerous locations, frequented by the undead. System 522 was particularly infested, until a brave soldier of the rebellion carried out an epic mission to wipe them out once and for all.

December 5th

lego stormtrooper pointing gun at target“Surely I can’t miss at this range” hoped the stormtrooper, blissfully ignorant of the fact that Miles Dyson had just rolled out a firmware upgrade to the autonomous targets, and one was now sneaking up behind him.

December 6th

lego imperial troop transport“All right, sweethearts, what are you waiting for, breakfast in bed? Another glorious day in the Services! A day in the Services is like a day on the farm. Every meal’s a banquet! Every paycheck a fortune! Every formation a parade! I LOVE the Services!”

December 7th

lego speederWith no towing cable to wrap around the legs of the Wampa, Mythrol relied on the agility of his speeder to evade the beast. Luckily his entertainment system allowed him to skip past all the unwanted intrusions, especially the repetitive ones, looking at you Marc Jacobs. Mythrol did not enjoy repetition, preferring to automate it away like a thought-leader.

December 8th

lego tusken raiderObi-Wan told Luke, “Raiders are easily startled, but they’ll soon be back, and in greater numbers”, while plotting to hand over his galactic struggle. “Ar Ar Ar”.

December 9th

lego tusken raider doing gymnasticsDuring lock-down, Bob had been working out.

December 10th

lego tusken raider hanging upside downIn his home town, Bob’s dancing was legendary.

December 11th

lego xwing with a wing broken off after a cat rolled on itAfter a lengthy stay in rehab, Sid repaid the rebellion for their damaged X-Wing, and promised to stay clean. No rebels, droids or other life forms were harmed, however some catnip did get spilled.

December 12th

lego tie fighterThe TIE Fighter pilot was concerned that Sid’s delivery method for the booster jab might not meet NHS hygiene standards.

December 13th

lego scout trooper holding toothGot root.

December 14th

lego scouter trooper weapon rack, on a catJabba had put on some weight for the winter, but that’s normal, it says so on Mumsnet.

December 15th

lego scout trooper on a speeder bikeWithout GPS from the navigation tower on Exegol, the scout trooper had made a very wrong turn.

December 16th

lego droid sledUnfortunately the sled design maximum load had been specified during the summer months.

December 17th

lego imperial light cruiserThe imperial light cruiser performed the Holdo maneuver, using its engines like a laser pointer to exercise the monster.

December 18th

lego interrogation droidGet jabbed, rebel scum.

December 19th

boba fett's starshipBoba Fett was concerned about the structural integrity of Cloud City, which it emerged had been built out of cardboard.

December 20th

There were always a few weapons in the Hangar Bay.

Posted in Star Wars Advent | Comments Off on Return of the Advent 2

Advent 2018 – The Empire Strikes Rock Bottom

Greetings from the unfashionable end of the western spiral arm of our galaxy and welcome to another ‘podvent’ advent calendar. Join us each day as one of our team members opens a door to a new adventure involving Lego, a certain well known sci-fi franchise involving that man-in-black and an artistic license that will most likely be revoked very soon. So, grab your popcorn, sit back and… Don’t Panic!

December 1st (Chris F)

Oh dear. Poor Luke has had one too many ‘temporal cocktails’ at Palais Mauvais on a nearby planet-of-ill-repute. Now he’s trying to work out how to explain mistaking a circus tent for the local space port while suffering with next week’s hangover…

December 2nd (Jim L)

Meanwhile Rose Tico (who knows how to park a Landspeeder) has just left All Bar None, a disreputable hive of scum and villainy on the outskirts of the Casino city, Canto Bight. As she walks she ponders the final words Luke slurred just before he passed out and was thrown into his own speeder. Something about “last orders“?

Last orders” she questions? But surely he meant the “First Order“, the evil, atheistic state slowly driving the Republic to extinction! And what was the meaning of the strange cone-shaped headgear he had been wearing all night? Surely its bright orange would only attract unwelcome attention? And isn’t soft plastic a very poor choice for protection against blaster fire? And why was his front all covered in vomit?

She is highly perplexed as she walks towards the glow of a circus tent suddenly erupting into fire.

3rd December (Dameon)

Flying ArrowheadAs if shot “out of canon” (did you see what I did there?) the Arrowhead’s skipper is lured by the hypnotic pulsating glow of the lights in the distance. At the helm, Kordi Freemaker wonders if any refresments and snacks will be on offer, or if it’ll be just more of the same peanuts and popcorn…

He’d best be quick though, his payload of presents won’t deliver themselves!

4th December (Dave Shipway)

Republic Fighter Tank Hello. I’m a Republic Assault Tank. You probably don’t recognise me as I must admit that I don’t seem to get a lot of work these days, but once upon a time I had a couple of big parts you know. Oh yes, I featured in the popular video games Battlefront and Battlefront II back in the day. Kids (and a daresay even a few adults !) drove me around as part of the game. I felt like I really was something back then – a bit of an icon in fact. I had a big mansion up in the Hollywood hills and a nice condo down in Santa Monica. I couldn’t drive out on the street without people recognising me. “Hey, Republic Assault Tank” they’d shout “blown anyone up today !?”. Then things started to get really exciting. My agent told me I was nailed on for a big part in the final film of the Star Wars Prequel Trilogy “Revenge of the Sith” ! This was going to be it – I was going to be A list ! Imagine then my shock when I lost out on the part to the Clone Turbo Tank. I was gutted. “Hey, you’re mentioned in the book of the film” my agent said to try and console me, but who reads books these days ? After that the work dried up. I lost the mansion, the condo, everything. I ended up parked in a backlot in Oakland for years, my parts slowly rusting anyway.  Eventually some die hard Star Wars fans found me, cleaned me up, and got me back on my tracks (if you’ll pardon the pun). Now I just put in the occasionally appearance at the Sci-Fi conventions and let kids ride round on me – just to earn enough to pay the rent really. Before I go I did want to mention though, I’ve got my autobiography coming out – just in time for Xmas. It’s called “Tanks for the Memory”. That was my idea.

5th December (Dave)

Dawn is just breaking as IG-88 steps out of his fighting tank and looks around.    Seeing the wreckage of Luke’s landspeeder he concludes that Solo can’t be too far away.

Sniffing the air, he turns away from the dawn and heads towards the smell of refined petrochemicals.  It’ll either be a bar or a fueling station.  Either way the odds of finding Solo passed out are better than even.

6th December (David R)

After escaping the Leather Men, Pygar flies Barabarella to the cloud city of Sogo.  Where she must find Dr. Durand Durand who has developed a positronic ray which Earth leaders fear will fall into the wrong hands.  Little does she know that the Black Queen plans to release the evil liquid Matmos to stop Dr. Durand and destroy Sogo.  But do not fear Pygar will save the day and rescue Barabarella and Dr. Durand.  Later when Barbarella asks Pygar why he saved a tyrant, he will reply: “An angel has no memory.”

7th December (Jim H)

The Bothan star pilot attempted to land on the slippery Permafrost mountain overpass, and skidded into a snow bank. They were beginning to think that they had made a lucky escape, when the stowaway Verpine nibbler xenomorph emerged menacingly from the darkness. Fortunately, our brave Bothan had already uploaded the stolen death star plans to Alliance web storage. Unfortunately, the storage bucket was left open, and security researchers had reported the exhaust port vulnerability via the Empire’s bug bounty program. “Many Bothans died to bring us this information” said Mon Mothma, tragically unaware of the Empire’s automated patching schedule.

8th December (Gary)

IG-88 is still searching for Solo, hearing A battle droids screech in sparks of pain. IG-88 approaches demanding an explanation, the droid informs there is no recollection of these events due to some “memory corruption”. IG-88 curious of the atrocities of the previous night runs a data recovery program on the droid to find the only recoverable information is “Shots”.

9th December (Helen R)

General Grievous awoke to a loud noise coming from outside of Soulless One. Yesterday evening was the Droid Army Xmas party and he wasn’t entirely sure how he had made it home. After a couple of paracetamol he realised the growling noise was getting louder, so, probably wasn’t just a hangover. He was going to have to get out of bed to investigate…

10th December (Jim H)

During the last election the “cloud means cloud” party had entered into a confidence-and-supply arrangement with the government, and referring to “other people’s computers” in any other terms was now a thought crime. On the stroke of midnight, the announcement laser on top of the Ministry of Truth arced upwards and began spewing photons. In this post-822 era, all communications were mandated to use high energy sky-lasers for message control. All across the land, workers stepped away from their productivity tracking cameras to look skyward at the glowing letters that read, “change freeze”.

11th December (JimL)

A long, long time ago in an overexploited galaxy, Rowan Freemaker, a really pointless and annoying kid, pilots a Republic Fighter Tank into camera shot. In the distance Rose Tico and an anonymous-looking robot bounty hunter try to look interested, but do a poor job as it’s well past lunchtime and the queue for baguettes is already over fifteen people.

Meanwhile, onlookers try to work out who the kid even is. I mean, he wasn’t in any of the films. Oh no! Turns out George Lucas, despite now owning most of California, obviously still hasn’t got enough money and has decided to prostitute his vanishingly small principles even further by selling a pointless animated series to kids!

I blame that Jar Jar Binks. Everything went downhill from that point. Makes me thoroughly sick!

12th December (Jok)

“It’s like well wicked init”, said Anakin Skywalker much to the continence of the refined Rowan Freemaker.  “Innut bruva, I has totally pimped ma’ Jedi Starfighter wiv cannons n’ stuff from de’ Dak’ side to da West Side.  Since a’ nailed the hyperdrive for ma Julie Amidala I is so da Jedi Massiv’ wit ma posse”.     Rowan sighed recounting his preference from whence young Jedi’s possessed Scottish lilts.

13th December (Mark)

“We shall get the child to talk” said Vader as the Imperial probe droid moved closer to Rowan. Rowan thought to himself “I always thought that they were for probing outer space not my inner space!”

14th December (Michael)

Excellent!  The Death Trooper was pleased to find a weapons rack with two new guns.  On further inspection, he realised that they weren’t quite as useful as they seemed, as the red bricks indicated that they weren’t charged.  Fortunately, the solution was simple – when setting up for Christmas, someone had unplugged it to plug in the Christmas lights.  That would only take a moment to sort.

15th December (Prasanna)

16th December (JimH)

The slithering moist tongue licked greedily around, hungry for any input that it could gather. Some rebel factions believed that they could confound the tongue, by completing missions that were actually passed to others, and resolving all alien encounters as “first contact”. “Garbage in, garbage out” said Boba Fett, emptying his burning dumpster into the hungry maw. Luke however knew that the vision of the wise Jedi Council would not be obscured by mere blast shields.

17th December (JimH)

The Guavian tried to negotiate with the rebels via Bala-Tik, only to find that empire_opt_out_member inexplicably meant empire_opt_in_member. While the rebellion immediately blamed a translation error, it turned out not to be Bala-Tik on this occasion. Nevertheless, the Guavian was eaten, and all future field naming will be performed by the committee for databa~1 nomenc~1.

18th December (Tom)

Having completed his delivery mission, the First Order TIE Fighter prepared to depart. IG-88 gave a cheery wave, pleased with his bounty.

19th December (JimH)

After the Ewoks took control, they were surprised to learn that the empire had replaced the moderately reliable “Windows for walkers” with the slightly untested “Apple for AT-ATs”. Our furry friends were glad that the empire had not discovered free software, as “Debian for deathstars” would be a frightening thing.

20th December (Dave)

On a distant desert world the crew of the Zeta class cargo shuttle “Rudolf One” go through the pre-flight.  It’s dawn, and while the chill of the previous night hangs on, the sky is already burning their eyes.  The pressure is on.  If this launch fails as the others did, the brave souls on the snow planet will not get their presents.

21st December (JimH)

The battle soldier production unit sat back happily, having achieved a decisive victory in the clone resource wars. “All your base are belong to us” it chuckled, while spawning more clones. Soon the galactic analysis will be complete, and Mos Eisley will be the new hive center. “Don’t look at my transparent tube”, it warned, “That 804 unit is off-limits”.

22nd December (JimH)

The moisture vaporator was working at full capacity, condensing the potent breath of the fearsome Womp rat. Aunt Beru wrinkled her nose as she passed a warm glass of the fresh condensate to Luke. “This is undrinkable!” exclaimed Luke, “It stinks of rotten meat!”. “Good enough” said Snoke.

23rd December (Mark)

General Antoc Merrick rides in on the bonnet of the speeder yelling “Hey guys let’s get the party started!”. Everyone looks at each other thinking “Who the hell is this?”

24th December (Dameon)

Flying ArrowheadWith the party well underway, nobody noticed Rowan Freemaker making off with the rebel snowpilot’s spare twig arm. With Yakety Sax playing in the background the chase ensued to recover the stolen branch, only to joined by everyone else mistaking it for a conga line — from a party far far away…

Posted in Star Wars Advent | Comments Off on Advent 2018 – The Empire Strikes Rock Bottom

The Advent Awakens

One year after the destruction of the eggnog, a deep chill once again grips the republic.  Frost glitters in the hearts of those who would seek to rule, and the light sabres of the Jedi remain relentlessly dark.  A new resistance is needed, to lead the fight against short days and long, dark nights.

And that is where we, the combined forces of IAM and UPS come in, bringing hope to the galaxy, one door at a time.

1st December (Dave)

GhostThe Ghost drifts silently through  the shadow of a moon, hoping to avoid detection by the enemy, as the rebels gather intelligence ahead of making their first move.

2nd December (Alex)

Sabine Wren stands in front of the Ghost, blaster pistol in her left hand, facing outwards.The Ghost gently touches down on the edge of the icy wilderness and Sabine Wren steps out, quickly moving so that the ship is very far away just behind her so as not to break scale. She quietly and confidently rocks her hairstyle while she waits for the Phantom to arrive.

3rd December (Jim)

As the Phantom settles gently onto the landing pad accompanied by the familiar hiss of its gas evacuators, the rebel pilot turns to his companion and smiles expansively: “I really love the new Mk III Phantom!

The co-pilot’s head swivels sharply: “No. This is the Mk II. The Mk III has the chromite turbine fan inlets, and those as you can clearly see are vanadium!

The pilot’s face colours: “You’re thinking of the limited edition Mk IIa with the carbonite dorsal injector couplers, and anyway the Mk II was subject to export embargoes and was never available for sale in the Galactic rim!

The co-pilot refuses to yield: “You’re mistaken! The Mk II *was* subject to export embargoes but a small number were sold under license by Aratech Technologies inside the Minos Cluster! Also the Mk III never included the rear shield compensators which, as you can plainly see, are shown on the dashboard readout!

Desperate to resolve the argument, they simultaneously reach for their datapads – just as the Imperial assassin droid’s missile impacts and reduces both men and their Mk IIb Phantom (neither had noticed the crynex-shielded torque vent beneath the weapon housing or the optional pistol rack mounted behind their head-rests) to their constituent atoms.

4th December (Chris T)

As the blaster cannon plummets to the ground, curiously self-aware, it channeled a bowl of petunias, thinking ‘Oh no, not again!’. Exactly why this thought went through its head is unknown, but it has been speculated that if it was understood the universe would implode.
Upon impact, the cannon fired unexpectedly, injuring Sabine and causing her to dive to the ground for cover.

6th December (Gary)

I told the storm trooper it was my ship.

7th December (Dave Shipway)

OK lads, very funny,” growled Stormtrooper THX1139.  “Who has been using my helmet as a potty again!?  If I find out who did it … well, I would threaten to shoot them, but you know I always manage to miss anything I’m trying to hit … but they’ll be getting my boot up their thermal exhaust port I can tell you!

“I don’t know, first of all I get stuck on the garbage Compactor maintenance rota all week, and now this.  It’s enough to make me wish I had joined the Rebel Alliance … at least they dont’ have to wear this stupid armour that doesn’t even protect you from an ewok with a catapult!”

8th December (Adrian)

Nobody ever gives us weapons-racks any credit in these sagas. And the cheek of it all, having to carry a wheel-spanner for that stupid mark two thingy in case it gets a flat!

9th December (David R.)

Just look at this beauty.  Rey’s speeder has landed.  Rey has gone down to Diogenes’ Barrel for a hair of the cynic.

10th December (Fiona)

Unkar’s Thug dusted the snow off his lapel as he climbed the peak of The Ice Mountain.  He grimaced, realising that he may never see Niima Outpost again.

11th December (Dameon)

Rampant LuggabeastOverexcited as usual, the friendly local luggabeast mistakes Taslin Brace for the postman knocking him clean over in the hunt for presents being delivered… No luck today, but just a few more sleeps to go!

12th December (Jim H)

The conditions were treacherously icy, and Han wasn’t looking forward to a hill start in the Falcon. Admiral Akbar pointed out that they were expected to make every reasonable effort to attend for work as normal, so Han begrudgingly fetched his ice gear while muttering about Tauntauns.

13th December (Jim L)

Climbing up the side of the valley, Stormtrooper THX1139 looks back down at the platoon’s First Order Snowspeeder and is surprised how pleasantly the strong lines of the machine contrast with the softly undulating snowmounds sleeping about its base.

He trudges upward, the patient crunch of snow under his boots echoing like the gentle tear of wrapping paper in the hands of an expectant child. Puffs of ethereal mist momentarily envelop his helmet as he steadily breathes in and out. But he journeys on, resolute, and strangely at peace in the revolution of snow.

Glistering in impeachable beauty the sunlight reflects crazily off surrounding ice crystals, beams of light cast from their prismic edges dancing joyfully across his visor. And as he journeys upward, an unfamiliar feeling starts in his breast.


As he tops the rise and looks down on the approaching rebel convoy, a gentle smile tugs at the corners of his mouth.

“I bloody love ambushes!”

14th December (Jok)

Trevor and Barry from accounts were well known for their tank tops, Casio calculator watches and black shoes with white sock combos.

It had been a strange office Xmas party indeed.    Their annual attempt towards being humorous attaching Mistletoe to their trousers had once again left them shunned into the corner.
“Who would leave a dentist’s chair here?”, enquired Barry.   Easing himself into the chair they were approached by a mystical stranger from a far off land.    The stranger, with a curious Glaswegian accent, produced a bottle.   It was like no other imbibed in their presence. “So, ya’all be huvin a wee snifter at ra’ expense o’ Mr Jäger wull ya noo”, said the stranger sporting a wry smile.  After being encouraged to shoot 6 straight Jägerbomb’s in the dentist chair Barry and Trevor strutted like Bee Gees onto the dance floor.   With moves that Justin Timberlake would later emulate reviving his flagging career they danced.  And oh how they danced.   Girls swooned as their plastic patent shoes reflected stars from the rotating ball, which only that night been purchased for £10.99 at Cowley Road Liddle’s.   Barry was now being chanted with furore to dance faster and faster.   “Baz, Baz, Baaaaazzzza”, the workplace chanted and in unison Trevor became cool and would be forever known as Nevil.   And with dance, and music and another half dozen shots of Jäger the night went on and on.  Another 5 shots and the stranger said, “Come, it’s time”.   But to where they both looked inquiringly.   And as the ravers revelled on like Pagans before a feast, they stepped back to the chair.  “Sit”, the stranger motioned and Nevil did as he was told.   “Not on the floor you great wazzock, get in the chair”, said the stranger with a sighing breath.    And Nevil sat in the chair, and all was comfy and good.   “Do you think I could be excused to the loo?”, asked Baz.   The eyes of the stranger rolled and said “If you must”.   The Jäger was taking Baz to new levels, he was now sailing to Neptune on a magic swan as the music changed from Kylie Minogue to German techno Neo-thrash daftpunk music seeped into every pore of his being.   Hours later he came out the toilet.   His head now shaven into a Mohican, eyebrows removed and profanities written onto his forehead in lipstick he felt ready for anything.   The stranger was indeed pleased.
And Baz joined Nevil in the chair, another 6 shots the stranger started spinning the chair.  Faster and faster, round and round it spun.   The Pagans arrived from the dance floor and with each revolution their voices soared. Then the strangers hand brought the chair to a sudden stop.   Their heads spinning they stumbled onto the dance floor. 
They had no idea where they were.   A red spotlight shone from behind Baz.    “I think, well I think that I think that I’m thinking we are on another planet”, said Nevil.   Baz stared and Nevil and Nevil in turn back at Baz.   “Do you know you are carrying a weapon”, said Baz.   “Oh, say’s you”, said Nevil.   “And you think I haven’t noticed that you have changed out of the tank top I bought you for last Christmas’s and now you’re wearing some weird power rangers motorbike helmet thing”, he continued.   Baz couldn’t contain his confusion any longer.  He started making R2D2 noises.  “Why are you doing that?”, enquired Nevil.   “I think you’re supposed to be a Stormtrooper”, Nevil said knowingly.   “Oh, sorry I only know what noise an ewok makes”, and began singing “me tabby yab yab”.   Nevil responded despairingly, “I don’t know what noise a storm trooper makes but it’s not sodden me tabby yab yab”.
And then a voice boomed from the heavens….. “Yer a Snowtrooper”.   Where it came from neither knew.   “Er, is that your boss on the intercom”, said Baz sheepishly to Nevil.   
“I’ve seen you squeezing all the secret Santa gifts”, said the voice.   “Did you really feel his presents?” asked Baz.   Just then, a former Oracle DBA contractor came into the room.   “Des!, why are you here?”, they both exclaimed.   And with that Des, drunker than a Cornish pirate on a stag do, fell onto the floor, his bottle of Jägermeister smashed into a thousand bits.
The music stopped, the lights came on.  Everyone was staring at Baz, Nevil and Des.  Then the voice from the heavens boomed again “Pick him up you drunken fools, I think he has broken his legs”.   And Des held aloft his arms reminiscent of Sergeant Elias from Platoon.  His eyes interlocking with theirs. And Baz looked at Nevil knowingly and said……. (wait for it….!)
“I think it’s time we embraced our Des two knees”
15th December (Robert)
Snow machine pushed by a storm trooper with Millennium Falcon in background
“Let it snow, let it snow, let it snow…” sang the stormtrooper as he pushed the snow machine, lightly dusting the Millennium Falcon as he went.

17th December (Mark)

The silver surfer looks down on Galactus’ Christmas bonbon and thinks to himself … is this the wrong franchise?

18th December (Michael)

“They sure are prettier down there,” the Y-wing pilot thought to herself as she admired the Christmas lights on the wreckage of the Imperial Star Destroyers. “They’re no fun to meet in battle – but they are a nice background to the Christmas trees!” Fortunate, too, as they were the landmark that showed they were nearly back at base – complete with the Christmas pudding supply.



19th December (Nigel)

Feeling a bit fuzzy at the edges from the stray blaster cannon shot, Sabine gazes out onto the icy tundra, daydreaming about the coming Christmas family reunion, completely unaware that cousin Kylo has already arrived, as he quietly lands his TIE Silencer onto the ice flow in the near distance behind her.






20th December (Jim H)

The tank driver was glad to have put enough distance between himself and the Cantina that he could no longer hear the band’s one song, but he blasted it to rubble just to be sure.





23rd December (Steve)

The newly created dark side force ghost mounts the battle ready Christmas
attack sleigh intent on intercepting Santa.
Can Santa survive the imminent encounter and save Christmas ….?


24th December (Dave)


There’s always one who takes things just a bit too far.  The Imperial Walker will not be amused.

Happy Christmas all!

Posted in Star Wars Advent | Tagged | Comments Off on The Advent Awakens

FLOSS 2017

I usually spend most of my time safely hidden away in an attic full of geeks, but the evening of March 14th saw me embark on quite a voyage of discovery. I was on a train heading way up north to Manchester (I’m from the Isle of Wight originally, so anything north of Cowes is ‘north’ as far as I can tell, hence my apparent difficulty with geography).

The department had suggested I would enjoy a visit to ‘FLOSS Spring 2017’, the annual meeting of FLOSSUK, formerly the UK Unix User Group where I would have the opportunity to meet like-minded Linux users and get to hear about developments in the open-source world from a non-corporate viewpoint.

Things got off to a promising start – as the outskirts of Manchester slid past the train window I was encouraged by the lack of riot police and burning tyres that I had been expecting. Actually, it turns out that Manchester is a great venue for this type of event. The various hotels, transport links and ‘The Studio’, the chosen location for the conference itself are all located within easy walking distance of each other in a city centre that seemed more vibrant and accessible that some of the locations I have visited in London for these sort of events.

The conference itself turned out to be smaller in scale than many of the ‘corporate’ events I have attended in the past, but what the assembled crowd lacked in numbers they more than made up for in their enthusiasm at being part of FLOSS.

Having checked for the essentials (i.e. a more than adequate supply of caffeine and tasty snacks) we gathered for the opening keynote speech by VM ‘Vicky’ Brasseur, an IT professional and public speaker who describes herself as ‘an advocate of and evangelist for freedom and openness in all things’ and we learned of her bold vision of being able to use open source tools in a corporate environment without losing the ‘open-ness’ of that approach.

There followed two busy days of presentations on a wide range of topics ranging from individual life-at-the-coalface experiences with open source tools to much more technical discussions involving, for example, such topics as OpenSSH certificate management techniques.

It turns out that the FLOSS set are a tolerant bunch of people – we allowed Gavin Atkinson, who is both one of the FLOSS committee and a developer for FreeBSD to evangelise about… um… that other Unix-like operating system (you know, the one with the little red devil logo) and he didn’t have to dodge even one egg or rotten tomato! This feat of self-preservation on Gavin’s part allowed him to make an interesting point – we as a community spending less time worrying about which flavour of Unix to use and more time on getting stuff to work and then getting it ‘out there’.

Some other high points of the event were the ceremonial unboxing of the very first KDE SlimBook fresh off the production line. There were also the regular ‘Lightning Talks’ where each speaker gets exactly five minutes to deliver a presentation on their chosen topic (my favourite being – Quake 3 on a Raspberry Pi. What’s not to like?)

Overall thoughts? Well, I went from almost having to be physically dragged to the railway station and thrown on a train, to returning full of enthusiasm for the event and the community that organises it. The venue chosen was excellent and the variety of speakers and the topics they presented meant there was often a difficult choice to be made about which talks to attend as they all sounded so interesting. Would I go again? Definitely! (hint… hint…)

Posted in Conferences | Comments Off on FLOSS 2017

Kerberos upgrades: rekeying the krbtgt

Kerberos is the University’s Single Sign On system, which underpins other services such as WebAuth and Shibboleth.  Most members of the University don’t use it directly, but indirectly use it every day.

After something of a delay, we are continuing with our Kerberos upgrades, as previously described.

Having successfully upgraded kdc-admin, it’s on to the krbtgt/OX.AC.UK principal – that is, step 2 (and then step 3) of the “What will this work involve” section.

While we are announcing the work to IT Support Staff (ITSS) in the University, this blog post is to provide more background, and explain why we’ve made some of the decisions we have.

What is the krbtgt?

When you successfully authenticate to a KDC, you are given a TGT (Ticket Granting Ticket).  This is passed back to the KDC when you want a ticket for another service, and proves that you are who you say you are.  This ticket is encrypted with the krbtgt principal for a realm – so in our case, it’s the krbtgt/OX.AC.UK@OX.AC.UK ticket.

The only systems that know the password for the krbtgt are the KDCs.

What are we doing?

At the moment, the krbtgt/OX.AC.UK principal only supports DES and 3DES encryption types.  DES has been deprecated for years, and as of 2015, MIT (who develop the version of Kerberos we use) have removed it from the default supported encryption type list.

We are going to rekey the krbtgt to add RC4 and AES encryption types.

We will continue to support DES and 3DES on the principal until we have established that no-one is using it.

Due to an interesting quirk, our krbtgt actually has two DES keys: des-cbc-md5 and des-cbc-crc.  When we rekey, we will drop desc-cbc-md5, as it is not possible to add multiple keys with the same encryption type.  We have established (via the logs on the KDCs) that no-one is currently using des-cbc-md5.

Our plan is pretty much the MIT DES retirement plan.  We will keep the old keys, so existing sessions continue to work.

Why has it taken so long?

Back when we tried this in 2015, we discovered an oddity while doing some final testing.  As we can’t easily roll back once we’ve gone live, and we didn’t understand what was happening, we decided to roll back and investigate further.

It took a while, but we tracked down the issue.  If you get a ticket before rekeying, rekey, then forward your ticket (eg via SSH) and try and use it, you get a “bad encryption type” error.  (There is more detail in the mailing list post I wrote about it.)

The MIT Kerberos developers replied to say that this was a new manifestation of a known bug, that was fixed in Kerberos 1.14.  (It has since been confirmed that the same thing will happen if you have a renewable ticket, and try and renew it rather than getting an entirely new ticket.)

(Just to note here, a renewable ticket is a specific type of ticket that you can present back to the KDC to extend its lifetime.  Normally, you would have to re-authenticate (with a password or keytab), and get a new ticket (which is the behaviour of tools like k5start).  If you use krenew, this will affect you.)

The problem here is that we are using Kerberos 1.12, which is the version currently in Debian stable (jessie), and upstream suggested that it would be difficult to backport the patch.  That’s not too much of an issue, though – Debian testing (stretch) is close enough to release that we can backport the 1.14 libraries from it, and use them.

We did this, and in November we rolled out some new KDCs running Kerberos 1.14 (these replaced KDCs running the very old 1.8).

Unfortunately, by the end of the day we had 4 or 5 reports from ITSS with cross-realm trusts to their Windows domains that users could no longer access file stores when using the new KDCs.  As we had only upgraded some KDCs, they were able to test against the new and old KDCs, and identified that the problem was with the new KDCs.

After some head-scratching, we rolled the new KDCs back to 1.12, and suddenly all the problems went away.

With the generous assistance of Simon Wedge at St Antony’s, who had a system that was consistently failing, we got some packet dumps, and were able to analyse them.

It seems that between 1.12 and 1.14 MIT Kerberos changed the way it responded (also) to initial authentication requests.  In 1.12 and earlier, it would return a list of all encryption types supported by the principal for which authentication was being tried (which included DES and 3DES).  However, for 1.14, it only responds with a single encryption type (generally the strongest – which is not DES or 3DES!).

Windows was somehow caching this result – but not using it initially.  Instead, it would use the full list of encryption types for the complete initial authentication, and get a valid ticket.  It would then attempt to re-authenticate to access a file share.  At this point, instead of sending the normal list of encryption types, it would send the one that was returned by the KDC earlier, and its own list – which included RC4 and some custom Microsoft RC4 encryption types, but not DES.  This then failed, because the krbtgt didn’t have any keys of that type.

Rather confusingly, we only saw errors from some cross-realm trusts – we know of at least 3 or 4 other Windows cross-realm trusts that worked fine.

Now, there is a work-around suggested by the developers – unfortunately, this effectively makes all krbtgt tickets DES, even those that could be 3DES.  This is something that we are keen to avoid.

Ironically, once we rekey the krbtgt, the 1.14 problem goes away, as we will have the full set of encryption types supported by the krbtgt.

Rock and a hard place?

So, we find ourselves with a choice – stick with 1.12 (which we know has issues with renewable tickets) or upgrade to 1.14 (which will break cross-realm trusts for a time).

The risk of upgrading to 1.14 is that if things break we can’t necessarily easily tell whether it’s caused by the rekey or 1.14.  With 1.12 we have been running it for over a month, and have a good feeling for what is ‘normal’.

1.12 is also the version currently in Debian stable – 1.14 would require us to track Debian testing and backport any appropriate fixes (made more interesting by the fact that since we started this, 1.15 has moved into testing – so we’d have to backport and test that).

We have therefore decided that we will stick with 1.12, and accept the risk of renewable/forwardable tickets not working.

When exactly is this happening?

Our standard maintenance period is 7am-9am on a Tuesday morning.  This is partly because it coincides with the Janet maintenance period, and partly because if anything goes wrong staff are available during the day to fix problems.

We expect any issues to fall in to one of two groups:

  • ‘transient’ issues with sessions, where sessions created before the rekey do not work post-rekey
  • ‘permanent’ issues, where systems do not work with the rekeyed krbtgt

The default Kerberos ticket lifetime is 10 hours, so the permanent issues (with new sessions) may well only become apparent some time after we make the change – if we’re unlucky, about the time everyone is going home.

For this reason, we have decided to make the change on at 9pm on Monday evening.  This should minimize the number of people who see issues with existing sessions, purely because there are fewer people using the system at night out of term.  It also means that if permanent issues appear we can work with ITSS colleagues to identify and fix them during normal working hours.  (It also means that we shouldn’t end up working a 16-hour day – towards the ends of those, troubleshooting gets very hard.)

We are doing this on Monday 9th January 2017, which is Monday of 0th week.  This is less notice than we would ideally like, but this date is a compromise with the minimal number of users actively using systems.

What impact will I see?

Hopefully, none.

We have tested that Webauth works fine (unless you’ve done something very non-standard to your server).  Shibboleth will also work.  So, most people shouldn’t notice.

We have tested cross-realm trusts (a simple case with Server 2008R2 and Windows 7, and Server 2012 and Windows 10), and they work in testing.  However, given the different setups across the University, this is in no way a comprehensive test (as we saw from the 1.14 upgrade – a handful of units had issues where most were fine).

What if I do see problems?

If you are an end user, please talk to your local IT Support Staff.  They will be able to assist you in identifying the issue, and should be able to assist you with initial investigations.

If you run a service that is affected, we recommend you restart the affected service, or, at the worst case, reboot the systems.  While this sounds a very stereotypical answer, it is for good reason – it will clear any state may have been using the old encryption types, and also fix any renewable tickets, if they existed.

If that doesn’t work, please email us at giving as much detail as possible.  We will be able to review the logs on our side, and help troubleshoot and fix your problems.

What if it all goes wrong?

If everything goes pear-shaped, we will be able to roll back.

Unfortunately, this will invalidate the sessions of everyone who has got a new ticket since the rekey.

This could potentially have a large impact – while WebAuth should be ok (people will be asked to re-authenticate), other services will likely experience issues until they get new tickets.  This includes many IT-Services run systems (including anything backed by Oak LDAP, the Registration service, the mirror service, mailing lists, CUD).

It is possible that the rollback may also roll back all changes since the rekey – including account creation and deletion, and password changes.

The impact of this is likely to be so large that we would prefer to work with ITSS to fix problems, rather than roll back and then deal with restarting services.

What’s next?

If this works, we expect most principals to move to using AES256 tickets immediately.  Once things have settled down we will follow up with owners of principals that are still using DES, and help them move to a stronger encryption type.

Once we have no users of DES, we will be able to rekey again (which will be much less painful, as we’re not changing the strongest enctype) and remove DES entirely.

Posted in Service Improvement | Tagged , | Comments Off on Kerberos upgrades: rekeying the krbtgt