Outlet to the Danger Zone

Chris Q had a reputation for being a bit of a maverick. He didn't make changes directly to production, dare the two-week-old butter chicken in the back of the lunchroom fridge, or even particularly like 1986's Tom Cruise / Val Kilmer vehicle Top Gun. But as part of an elite development team that had split from Government Department's mainframe dev group, Chris couldn't help being branded "other". When he walked by, the old-school mainframe developers whispered: there went a guy who thought dangerously out of the box.

The original reason for the split was called The Internet. As the Thermodynamic Arrow of Time dragged them relentlessly towards the heat-death of the universe, Government Department needed a web presence and a modern intranet. While the budgetary committee would have been perfectly happy to somehow run the new system on the Department's existing mainframes, sanity prevailed (this time) and a new-ish PC server machine landed on Chris's doorstep. Since a majority of the senior-most developers wanted nothing to do with the newfangled equipment, the PC / Server Team was formed. While having perhaps not as much experience as the mainframe developers, Chris and co. knew their server needed an uninterruptible power supply and regular disaster-recovery testing. Every quarter, at a scheduled time, they would pull the plug on the UPS and ensure the server shut down gracefully before the batteries died. And the server always did.

Everything was fine until that pesky Arrow of Time saw fit to move Government Department forward a couple years and into new offices. The "mainframe" support team had since added (or, some said, been forced to add) modern servers to their roster, running things like their new helpdesk system. Though the PC / Server Team and their senior counterparts remained separate in mind, body, and certainly in coding conventions, their hardware huddled close together in the new office's compact server room. All was well... for a few months. When quarterly testing time arrived, Chris pulled the plug on their UPS. As expected, their server logged the UPS warning messages and did a proper shutdown. Once the batteries were flat, Chris would record the maximum emergency window provided by the UPS, and he was just reaching for his logbook when he heard the screaming.

The mainframe support team poured into the server room, looming over Chris. Their systems were crashing! What had he done?!

It was a tense moment, but Chris - that maverick - answered a question with a question: had they plugged a server into his team's UPS? They had: three of them, in fact. After all, the three empty outlets on the power supply, blaze orange as they were, looked so inviting. And, Chris - thinking outside the box - followed with another question, had they installed UPS-monitoring software on those servers? They had not. So while Chris's server had realized it was on battery power and began its shutdown sequence, theirs had cheerfully dawdled until darkness fell.

That was how Chris wound up giving Government Department's senior developers a riveting Powerpoint presentation on how to properly use a UPS with their servers. After the talk, one of the mainframe gurus stopped Chris and shook his hand.

"You're still dangerous, maverick... but you can be my wingman any time."

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Featured Comments

Chris Q (unregistered) 2013-07-22
Being the maverick named in the (dramatised) account above, perhaps I should make the following points:
1. Without crawling under desks, it was not possible to see what was plugged into the UPS - all I knew is that our server was.
2. Rather than physically unplugging the UPS, all I did was switch off the power outlet that the well labelled UPS power cord was plugged in to.
3. The maximum battery time was measured to have some idea of the battery condition. If the time available dropped markedly, we would assume that the batteries were in need of replacement. The first time we saw a markedly reduced battery life (though still sufficient for a graceful shutdown of our server) was the occasion mentioned above.
4. I contend that the WTF here was not mine, but rather the idiot who plugged production servers into a piece of kit that was, to him, unknown and undocumented. It was documented in our procedures, and the other team was aware that we looked after our own kit.