It's Raining on the Robot

On the way to the data vault, Dave and his coworkers tried to list every rain-related song they knew. Here Comes The Rain Again was an easy one. Ryan, raised in the nineties, offered I'm Only Happy When It Rains. Justine tried to get out in front of the competition by rapid-firing November, Purple, and No. Thad, veteran of a hundred karaoke battles, offered Blame It On The Rain.

But none of them had heard It's Raining On The Robot before.

That's all the caller had said: the Red Phone, the one which never rings, had jangled to life and the whole support team had just stared at it, trying to remember what it meant. The voice on the other line yelped only those five ominous words before hanging up, so the entire team scampered to Building Three and down to the sub-basement to investigate. Turned out the caller hadn't been speaking in riddles: there was a steady fountain of water pouring through a vent hole in the ceiling, puddling on the roof of the venerable STK Powderhorn silo, and starting to drip into the tape libraries within. Whoever had stopped by the silo and noticed the downpour was long gone, hopefully to find their STK consultant, so the support crew got to work on cleanup. Justine found a couple mops and buckets, and she and Ryan began to wrangle the growing pool on the data vault floor, arguing over the lyrics to Raining In My Heart as they went. Dave and Thad rolled their eyes and headed upstairs to find the cause of the leak, though Dave couldn't help himself from humming a few bars of Who'll Stop The Rain as they sought the right room.

Dave found the culprit when he happened to glance into the decommissioned server room off the hallway above the data vault. Building Three was the oldest building on the company campus, and it had central heating; an odd choice for a datacenter, but so it goes. Word had recently come down from on high that the radiators were to be replaced, and this old server room was first on the list. The hot-water pipe servicing the radiator had been cut, a simple job of freezing the pipe with dry ice, making the cut, then sealing the ends before it thawed. The plumber was some kind of maverick, apparently, because he'd elected to insert a bold new "taking an extended lunch break" step into the process. When the ice melted, the radiator had spilled its guts, washing twenty years of accumulated dust and grime from under the false floor into the data vault below.

Dave and Thad returned downstairs to find Carter, the STK consultant, on the scene. The Powderhorn was a tricky beast, so he was kept on retainer and it hadn't taken long for him to show up with a heavy-duty blue briefcase marked "Emergency Use Only". Justine and Ryan were soaked to the knees, leaning on their mops as they watched Carter carefully scan the rows and rows of tapes in the silo, removing and setting aside any that showed evidence of wetness. When he was done, he set the briefcase on a table and reverently unlatched it to reveal... a hair dryer and an extension cord. So armed, he proceeded to return his moistened charge to its former, working state. With disaster averted and only a few dozen tapes destroyed, the support team dispersed to let Carter finish his work. When Dave and Justine popped by Building Three days later to confirm that the robot was humming dryly along, they saw that Carter had implemented a new emergency protocol: leaning in the corner of the vault was a big roll of sheet plastic, marked "Emergency Use Only".

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!