Merry Christmas! It Doubles as a Saw Horse was originally published on November 3, 2006.


A little more than a decade ago, John Rudd was a Computer Science student at the Georgia Institute of Technology. He worked closely with the university's IT department and played a vital role in the creation of a new state-of-the-art data center: he unplugged and labeled cables before the movers relocated the servers and plugged them back in at their new location. There was one thing that struck John as being a bit different: the data center wasn't fully built yet.

For the most part, the data center was complete and needed just a bit of interior work. This wasn't a big deal, though, as the contractors built a wall of plastic sheets around the area to be worked on and draped the servers with a giant plastic sheet, just to be safe. Some of the network administrators were worried about heat build-up, but after a few days of carefully monitoring the servers and watching them run cool and without issue, they worried no more.

Then, all of a sudden, one of the servers crashed. Hard. As in, a completely non-recoverable major disk failure. To make matters worse, it was the core file server and vital to the university's information systems. The network administrators had a new disk overnighted from the vendor, rebuilt the server, and restored all of the data from backup. The university was up and running again within 48 hours.

Then it happened again: major disk failure, non-recoverable. Overnight a disk, rebuild the server, and restore the data. You know the drill.

And then it happened a third time. The vendor didn't overnight a disk this time, they overnighted a technician and a disk. I don't think they shipped them in the same box, though. The technician confirmed that the disk itself was bad - not the controller, power supply, or anything else. On a whim, he cracked it open.

It was physically damaged alright. The disk heads managed to cut deep, jagged groves right on the platter. It was completely visible to the naked eye and quite a site to see. But no one could figure it out. Disk heads just don't do that. Well, unless they're possessed by some poltergeist or something. But the building architect was pretty sure that they didn't build on top of an Ancestral Holy Land / Orphanage Burial Ground.

A few days later, one of the network administrators just happened to be walking through the data center. He saw that one of the workers set up a sawhorse, laid a board across it, and set the other end right on top of the server's disk enclosure. The worker was just about to make a cut with a saber saw when the admin stopped him. Apparently, the crew had brought only a single sawhorse to the job site and, after finding a convenient sawhorse replacement, saw no need to get a second one.


[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!