| « The Incrementing Bit Column | The Key Lock Box » |
“HEY! YOU!” barked a frantic and unfamiliar fellow in a frumpled collared shirt who barged into Daniel’s cramped little office.
“Are you running...," he asked while consulting a clipboard, “Google Desktop??”
Daniel started to formulate an affirmative answer, but before he could complete the word “Yes”, his interloper interjected “WHAT!? TURN IT OFF!! It’s crashing the proxy server!!”
“Wait, what?” jumped in Daniel, “Crashing the proxy? It checks Gmail like...every 9 minutes! And just who the hell are you anyway?!”
Daniel’s adversary narrowed his gaze while tapping his clipboard on each word - “I said turn it off, so TURN. IT. OFF.”
“I find this all very hard to believe. I’m going to speak to Dave.”
Daniel was on pretty good terms with Dave. As the company’s Lead (or rather, sole) Network Technician, he felt Daniel’s pain considering that Daniel was the company's Lead (again, which meant only) Developer and they helped each other out when they needed it, so it was a bit of a shock that he would get dinged without so much as a heads-up.
An inherited “crapplication” was something that Daniel could accept, however for something boneheaded as this, he needed an explanation.
“Tell me this then - do you mean to say that the office network that you designed is so fragile that 2kb of traffic a MINUTE was enough to take it down?!”
“Yeah, sorry about that man,” Dave began, “didn’t mean for Jim’s IT goons to get ahold of ya. Things just kind of went to hell in a handbasket real quick, ya know?”
Daniel glared a little as Dave leaned in and whispered “It sort of, well, um, wasn’t actually the network.”
As it turned out, there was a little secret in the company.
At some point in the past, someone had "guessed" the login to the company's web server and had wreaked quite a bit of havoc.
In searching for a solution, a sweet talking snake oil software salesman fed into the Owner’s paranoia of being hacked and sold a whiz bang TCP-IP monitoring and security suite, custom-built server, service contract - the works. Since the owner wasn't convinced that the break-in wasn't internally motivated, the monitoring package was configured to record EVERY HTTP request originating from within the company's network in real time to a flat file on the network's web proxy server. The flat file was then processed into a PDF with the "bad" sites in red. This process took 10 minutes per employee and was run once per hour, every day, and was reviewed daily by IT management to ensure compliance.
Dave went on to explain that attempting to write the Google URL to the file every few minutes would lock the file (which was in use by the PDF process) and crash the whole thing. There was no recovery, and the proxy would simply hang with the Gmail URL being flagged as the "culprit" when really it was just terrible, terrible design.
In the end, Daniel removed Google Desktop, which was a bit of a pain for him, but it wasn’t all bad though. In fact, as a result of the experience, he gained some good reading! The aforementioned PDFs were stored in a shared network drive where any employee could easily read up on the owner's obsession with AdultFiendFinder, or the IT Head's hours spent browsing YouTube.
|
As the author of this particular wtf, I thought I'd clear up some confusion:
ALMOST right. The PDF reports were generated once a day, from midnight to 1am. The only machines making web requests were the development machines, since they were left on and had google desktop running. If you made a web request while your log was being parsed, it crashed. While I agree with that wholeheartedly, a 16 year old kid with a clipboard (whom I'd never seen before) barged into my office without knocking and demanded I uninstall software. Technically, given our security measures and the color of his access badge, instead of calling Dave I should have called security. If someone I KNEW had demanded I turn it off, I would have done so, and then asked questions. Sadly, there were nearly 2 dozen developers at this place, plus 4 network guys, and a horde of generic "IT" guys for actual computer fixing for the non-technical staff. I simply left them out of the story because they weren't relevant. I was one of multiple developers working contract roles, plus a dozen or more full-time developers. Searching our outlook mailbox would take 15+ minutes. Google desktop was used almost exclusively for email indexing and to-do lists. All sadly true, and quoted from my email conversation with Alex.
And the answer is... DING DING So yeah, the place was pretty terrible, from a tech standpoint. I'd like to voice my appreciation for all the NON-stupid tech people at the company though, the only reason it's running at all is because of all the brave soldiers who kept their heads down and worked around all the nonsense to keep things running. |
| « The Incrementing Bit Column | The Key Lock Box » |
