Bruce W's employer was best described as The Mega Bureaucracy. It's the kind of place where it takes twenty weeks to provision web servers, because of the number of forms, checkpoints, and management sign-offs involved. The Mega Bureaucracy did all of this because it kept their environment "stable", and equally important, "secure".

Speaking of security, the Mega Bureaucracy needed to expand its offices, and went out and constructed two new fourteen story office buildings which would serve as their headquarters. These offices needed to be validated for security, and Bruce was invited to be on the team that would perform the assessment. The first area they visited was the mailroom which served both buildings.

Of course, there was great concern over mailroom security. You didn't want things being added to or removed from the mailstream without going through the right process. This meant that the mailroom needed physical access controls: only authorized personnel should be allowed into the mailroom. Everything entering and leaving the mailroom had to pass through a control point.

Bruce and the rest of the assessors did a walkthrough of the mailroom space, and verified each of those steps. As one might expect from the Mega Bureaucracy, a lot of overhead and control had been added to the mail-handling process, but the result was that the mail wouldn't be easily tampered with by unauthorized users. Satisfied, the tour group left the secure side of the mailroom, and clambered into the elevator to examine the next floor of the building.

Bruce noticed that there were some stragglers, so he hit the "door open" button in the elevator. Except this was an elevator with doors on either side, and Bruce had actually hit the door open button for the opposing doors. They dutifully slid open, revealing the secure side of the mailroom. Anyone with access to this elevator (which was anyone in the building) could access the mailroom by simply pushing the "door open" button, circumventing the many, many security checks that the Mega Bureaucracy had put in place.

This was, as one might imagine, a problem. Bruce and his team raised the problem, which triggered a barrage of finger-pointing, name-calling, and extreme blamestorming as every entity involved in designing the building wanted to throw the other entity under the bus. The company that constructed the building claimed they built it to spec. Mega Bureaucracy claimed that they had not. The team that approved the design of the building claimed that their spec included these security checks, the elevator installer was emphatic that no such thing had happened. The refrain from everyone involved in construction was: "There is no problem, we built the best buildings, your expectations are wrong."

This is where the Mega Bureaucracy fell apart. Their entire bureaucratic system of command and control was entirely built around preventing these kinds of situations from arising. The number of signoffs and checkpoints was to prevent failures. As it turned out, there was no bureaucratic system to recover from failures. Instead, everyone just blamed everyone else for not having checked the right boxes, and the building started operating with a mailroom that anyone off the street could just wander into.

With no bureaucratic solution, Bruce took matters into his own hands. He called the building manager. The building manager cut him off with the refrain: "We built the best building."

"That's fine," Bruce said, "but why don't you meet me in the elevator in ten minutes, so I can show you what's happening."

Ten minutes later, Bruce met the building manager in the elevator. Five minutes after that, the building manager sent out an email announcing that they'd be adding a card reader to the elevator to control access to the mailroom. All in all, from the time they identified the problem to the time when it was finally resolved was brief, by the standards of The Mega Bureaucracy: three months.

[Advertisement] Otter - Provision your servers automatically without ever needing to log-in to a command prompt. Get started today!