- Feature Articles
- CodeSOD
- Error'd
- Forums
-
Other Articles
- Random Article
- Other Series
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
Plan: to go to Mars with a drill
Admin
Admin
Admin
By the way, I have seen electricians work on outlets without shutting off the electricity.
Admin
Admin
This one makes me think back to where I was working a couple years ago.
The company was expanding so it was determined that they needed a larger data center (when the SAN's were almost at full capacity and all the server racks were full and no room at the data center for more server server racks or anything, oh by the way the electrical circuit was maxed and several mission critical systems didn't even have ups backup).
Senior mismanagement at the company decided that putting the data center in a different facility was out of the question, moving the servers to a temporary location so they could safely expand the current facility was also considered out of the question, sooo it was decided that they would do an "in place" expansion of the data center.
The project was finally completed after several months of outages due to things being inadvertently unplugged, bumped into and the servers getting to breath and choke on gypsum and concrete dust while wall demolitions were occurring.
During that era, outages like the one described in this wtf were almost a weekly occurrence. It's amazing the place is still in business. (I'm quite happy to no longer work there. Even in this economy.)
Admin
The story can be condensed to: Facilities decides to install a molly guard. Electrician screws up and nobody told IT. They route server room into a different circuit. The end.
Admin
Admin
The real WTF is that this is some of the worst writing I've ever witnessed.
Admin
The real WTF is that this is some of the worst writing I've ever witnessed.
Admin
If you drill any further into the power supply, I believe the problem will solve itself.
Admin
Admin
Insane editing like in this article irritates me. The story should be good enough to stand on its own without extreme editing. Adding in random gags everywhere just for laughs takes away from the core story, and makes it feel fake and lame since it's obvious that parts such as the Lady Gaga music aren't actually in the original submission.
Articles should be edited to fix up spelling and grammar errors, improve the flow of the writing, etc... not completely hacked up in the editor's own creative way that makes the original story nearly unrecognizable.
Admin
Admin
The problem with "accessible" big red switches is that they are accessible. What needs to be done is have a "dye pack" that goes off when the switch is activated. Then EVERYONE will know (probably for a week or so) who did the dirty deed. It may not prevent PHB's from mischief, but it WILL let everyone know who the idiot was/is.
Now to patent the idea (you heard it here first!).
Admin
A woodchuck would chuck as much wood as a woodchuck could if a woodchuck could chuck wood.
Admin
How many woodchucks would a woodchuck upchuck if a woodchuck could upchuck woodchucks?
Admin
People have been complaining about this for years, but nothing ever changes.
Admin
TRWTF: if they followed your suggestions, it would bore me and everyone else straight off the site.
Admin
Wow, I concur that the writing in this article was horrible. I have taken the liberty of fixing it for you. You're welcome.
A chili wind went up Simon's back. A deep rumble rattled his brain in his skull. He glanced behind him to see Noel looming over him. "Simon," said Noel with watering eyes, "please don't eat chili for lunch again!"
When the trained bear that doubles as your IT manager asks that question, it's a bad thing. Realizing that you work for a trained bear is even worse. Simon tried to ping Exchange and failed. He then tried to ping an app server. And a database. And the outside world. All dead. "They're all dead," pondered Simon, "It must be the apocalypse!"
Now that they were dead, about a thousand zombies were about to discover that the only IT asset they could access were their desktop computers. Shortly thereafter, the phone next to them was going to ring. The person at the other end would likely scream "braaaaains!". And then it would ring again. The ringing and screaming and brain coveting would continue until Bruce Campbell showed up.
They hustled passed corpses to the server room, with Noel's bulk clearing a path through the corridor. It was hectic, and he may have trampled an undead intern. In one corner of the server room, a suited man was jumped by an unholy horde and bitten in the ear. The server room was so quiet that the two could actually hear his screams from across the room.
"What have you done?" Noel rumbled again. The stink of decaying chili scared away the zombies, buying them some time. However, the ominous wave of digested beans failed to penetrate the shield of Lady Zombies the electrician had been surrounded by. Noel closed on him and repeated the question, with more volume and a heavy hand on his shotgun.
Once he had rescued the electrician, he didn't wait for an answer. "I will tell you what you have done," Noel said, "you are cut power to the room."
"I… I'll fix it…" the electrician managed to croak out despite festering zombie bites.
He set to work. Noel set to work as well by providing a profanity laden commentary on the number of zombies. Simon just wreaked.
The electrician worked through the tirade, which says good things about his saving throw versus intimidation, even if his saving throw against undead left something to be desired. With a sigh of relief, Simon broke wind again. Green fog and the sounds of whirring filled the room.
Then the zombies stumbled back into position. Simon rattled back in retaliation. An excruciating sounding chomp came from an electrician somewhere in the room.
"I'll fix it!" the electrician squeaked. After a frantic adjustment, he flipped the breaker again. The show repeated itself, although the encore added some zombies from UPS. The third time the electrician reached for the breaker, Noel nearly broke his hand off and replaced it with a chainsaw.
"Hey, electrician, where are we? " Noel asked.
"Well we just crossed the Tennessee border..."
Noel ignored the electrician and turned to Simon. "I think I'm gonna heave." Simon threw up, and fifteen feet away, he saw the spare rib and seven-bean-chili that had been eaten hours earlier.
Since the electrician didn't have a mop -- and since Noel's throwing up wasn't up to the task -- Simon clambered on his back. One egregious violation of the health-and-safety policy, a round of ammunition, screaming and cursing pushed the undead back, although a few HDDs were trashed. Noel's blood-pressure receded, and Simon drafted a thoroughly plausible explanation as to why there were shoes and zombie heads scattered throughout the server room.
The root-cause for the debacle was traced back to Building Facilities. The server room, normally accessible only to sever admins and the facilities manager, contained an emergency zombie-repellent switch. Despite the fact that all of those people were quite clear about the purpose of the switch, and that it was clearly marked, someone with pointy-hair had deemed it an "unacceptable" risk, and decided to install a cover over it. After all, if left unfixed, someone might cut off power to the server room and start zombie armageddon.
Admin
You don't like the way articles are editied on TDWTF? Go start your own web site.
You don't understand Electricians, drilling, and the story? THAT'S HOW STORIES LIKE THIS HAPPEN. People who think they have a good grasp of the way real things really happen give sensible directions to sensible people, and the next thing you know, all the servers are down.
There are a lot of things you don't understand. Learn to deal with it.
Admin
Poor quality of posts on WTF of late. WHat's happened ?
Admin
It seems disturbingly plausible to me. The details maybe, but the outline is entirely plausible. But then, I work for someone who has a 5500kVA UPS with a pallet load of batteries attached and until recently was running the whole server room off one phase of a three phase UPS. Why? Because only one phase had "jug plug"(sic) outlets, the other two had 16A C17 plugs that no-one had ever seen before, so they weren't used. They also assumed that all the "jug plugs" were the same, so ran a couple of extension leads off those to power the rack. Via C13 to three pin adapter cords, of course. Unfortunately both sockets were on the same phase. It was only after the last new toys was added, the circuit breaker blew and the whole system went down that someone asked WTF. Specifically, the new guy (me) still in his first month looked at it, downloaded the manual, found the right page and said "are you fscking morons? Don't buy a new UPS, use the other two thirds of the one you already have". So we paid a couple of hundred bucks for two new C18 to lots of C13 power boards and now the UPS load is at ~50% on all three circuits. Plus a few things that weren't on the UPS at all are now one it... like the digital asset management server and disk array.
Admin
except for the part where the guilty engineer went strangely missing and Simon has a new toolkit...
Admin
The real WTF is that if your business is so crucial to servers being up 24/7 you have an alternative off-site backup server room that can run if you need to cut power to your main supply for any kind of maintenance.
If you are able to handle downtime at weekends (as is the case for many smaller offices) then do it then, not in the middle of a working day.
Admin
Hi,
I am an electrician. Some notes:
-Brendan
Admin
I've higlighted TRWTF for you.
Admin
Why do you keep using that word? I do not think you know what it means.
Admin
Heck, I replaced all the old 2-prong outlets in my house with nifty, newfangled 3-prongers while the whole house was hot.
Remember what's hot (other than that teen queen tanning in the neighboring yard) and what's not (your mom) and make sure they never meet.
Admin
I joined. Even without any more participation than just reading stuff that seems to be enough to drag a perfectly good site into board meeting levels of interesting content.
Admin
Admin
If you go back into history, you will see that (unlike Remy), Alex has a sense of humor; and when he makes a humongous screw-up, he's man enough to go ahead and promote some humorous comments pointing this out.
Remy, on the other hand, ignored all users, many requests for comment promotion, and put a stupid, unfunny meme as the promoted comment. I hope he gets demoted or at least gets a serious talking-to by Alex.
Admin
Admin
Admin
Admin
Some day, I have to write up the time that the construction workers carefully wrapped all of the running servers in plastic so no dust got into them.....
Admin
I think the main complaint here is that the editors are putting way too much of themselves into the articles. The fictional elements don't enhance the stories. Instead, it's quite the opposite. If the editors would just clean up the submissions rather than making them up (which presumably would be less work), the site would be much better, and the fact that they don't is frustrating.
Admin
The only author I have a problem with is Remy.
Admin
I find it odd that everyone wants to blame the electrician for the problems that occurred in this story. Two different problems were involved: powering down the circuit while the systems were up, and not being able to power the circuit back up again without overloading it.
The first of those problems occurred because the Facilities Maintenance department didn't tell the IT department what they were doing. If the IT managers knew about it and were involved (as they should have been) in scheduling when the work could be done, then they could have had their systems shut down gracefully before the circuit was taken down for maintenance. The electrician was most likely an outside contractor. It's not his job to make sure all the internal communications take place as they should have. He was hired to work on a circuit, given access to that circuit and the go-ahead to begin the job. Then he got bawled out because the department that hired him hadn't cleared it with the other departments that were impacted.
Then there was a second problem that occurred when the circuit was brought back up and all the servers overloaded it with the power they needed during startup. That's clearly the fault of whoever decided to plug so many servers into that circuit in the first place. It's most certainly not the electrician who did that. It's the fault of the IT managers (most likely Noel) who hadn't paid attention to the load capacity of the circuit when they deployed new servers.
So they had an error by the IT department and an error by the Building Facilities department, and then it's the poor electrician who got stuck in the middle who got yelled at about it, just because he tried to do the job he'd been hired to do.
Admin
Admin
How thoughtful of them. Did they also remember to shut off the air conditioners to prevent the dust from blowing around?
Admin
Admin
+10
Admin
I don't get this one.
Admin
Wow! Coming up with another electricity-related pun is going to be tough. But I'm sure if I put my mind to it I conduit. Just don't gauge me too harshly.
Admin
I have a big red button in the computer room right behind me. I sure would like to push it.
Admin
on red buttons: many years ago, in the days of big iron, 7am to 8:30 am was time for systems programmers to try out the next upgrade etc. There was an emergency power off button, fairly low down near the door to the operators inner room (space for 6 or more people)
One day, the Senior System Programmer brought his three year-old to work (during the day) and showed him where Daddy worked. Took him to the machine room (this was in a university, no high security). Talked to the operators about something. Three-year old wanders off, sees inviting red button and presses it. Everything powers down, for 15 minutes. Much irritation. No work lost, just a little delayed.
A few weeks later, Senior Systems Programmer is ending his systems session, and restarting the regular system at 8:25. Is presented with option "do you want to format the queues ?". Chooses yes. System burns all input queues (from 20+ distributed RJE sites) and output queues (the results of the overnight runs on the machines, vital to sundry researchers).
First incident caused a circular from management that this was not a nursery, children not be admitted etc. Second just resulted in derision.
My view was that there was nothing wrong with letting SSP junior wander about, but the real menace that needed supervision was SSP senior.
Admin
SOP, especially if he doesn't have (or trust) wiring plans.
Admin
I've seen the cups you guys fill with the insipid swill you call "coffee", and they're so big that the time it takes to fill them every day adds up to months of vacation anyway.
Admin
we have had an AC guy lop off a cable in the server room. when we came to investigate, he was completely disappeared. And the cable he cut was BETWEEN the UPS and servers. I believe the story just because i have seen lots of REALLY stupid service people...
captcha: tego - trainee electrician go away