- Feature Articles
- CodeSOD
- Error'd
- Forums
-
Other Articles
- Random Article
- Other Series
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
Opps. Oh, I mean UPS.
Admin
Kind of silly if you think about it. A UPS powering a whole farm of servers (which are designed for two) and another UPS? Seems a bit of a ridiculous mistake to make.
Admin
The requisition was filed next to the Ark Of The Covenant.
Admin
The 9V batteries in the picture appear to be wired in series, which does not provide redundancy. Although, if surreptitiously applied to a computer mouse, they would provide hilarity.
Admin
I see the problem: Let's say the servers are in Rack A and Rack B. UPS 1 was providing backup power to Rack A. UPS 2 was providing backup power to Rack B, and primary power to Rack A UPS 1 was plugged into UPS 2, but UPS 2 couldn't provide enough power.
They needed to plug the primary power for rack B into UPS 1, and UPS 1 into UPS 2. Then they'd have had all the servers on two UPSs, and each UPS on a UPS. As a bonus, if the power failed, both UPSs would still have been charging while they were powering the servers, so their batteries would not have run flat.
Admin
Yep - ever welded with 9V batteries in series? You can scare the hell out of someone doing it :)
Admin
I didn't really understand this, but I want to comment anyway.
Power to the people. TRWTF is UPS.
Admin
Yo dawg I herd you like UPS so I plugged a UPS in your UPS so now you can power your servers while you're powering your servers.
Admin
I blame Hubert. He checked the UPSs after the first crash. He should have seen how everything was plugged in and fixed it then.
Admin
Whoa, whoa, just be careful you don't feed the supplies into each other because you might create some kind of perpetual motion machine. Our corporate policy is to obey all the laws of thermodynamics.
Admin
It was too dark.
Admin
I love how the article is written. Points for Remy.
Admin
Message left on server: "UPS Delivery Problem NR 27743"
Admin
Sadly, this is extremely relevant to the topic at hand.
Admin
I agree. Why is Hubert involved? Where are the people that setup the UPSs?
He is either responsible for the server room or he is not. If he is, then he should be making an issue of the problem and fixing what he can. If he is not, then he should stay away from the server room.
Where is the source of this story?
Admin
Admin
I know nothing about baseball, but I'm pretty sure you're wrong there... (http://en.wikipedia.org/wiki/Innings)
Admin
Admin
I blame the system.
Admin
Admin
TRWTF is top men. If you want this handled, get a top woman.
Admin
Where I used to work the LAN Admin set up a new server which had 3 redundant power supplies and plugged all three into the same surge suppressor. Oh and the surge suppressor had a reset/trip, so that when the power went out, you had to manually reset it. At least they plugged that into a UPS.
Admin
Admin
Admin
Yeah, because we've never heard of anyone being in a situation where his responsibility outstripped his authority.
Admin
For those too lazy to view the source: click on "Poor little UPS #2".
Admin
Admin
Heh. Ours is much nicer, as befits a company that has the sort of servers that MUST be shut down in a controlled manner. Every wall outlet in the server room runs to a huge battery backup that can handle everything for about 15 hours.
We had an outage a few years back, knocked out the whole building. I got up when my pager started making frantic sounds, checked the power, went back to bed, got up an hour later, checked it to make sure it wasn't nosediving...No worries. Plenty of time.
At about 6:00am, my pager screamed again. Different tone. It's the 25% power warning. What the hell? I pulled on some clothes, and laid a patch pulling out of the driveway. Halfway to the building, pager screams again. THERMAL WARNING. One of the batteries must have caught fire or something! I slammed the pedal to the floor, and tried to keep from hyperventilating.
I got to the building, jumped out of the car, sprinted for the door. Took the stairs a landing at a time, and kicked into the server room to find...That a moron coworker had allowed some customer service reps to set up shop in the server room.
(Pause for people to say...WHAT?)
Yes. I suppose the logic went as follows: "Well, they need to do their jobs, and theres only one place in the building with any power, sooooo." The practical upshot was that he'd lugged a few desktops in, and plugged in a few phones, and then left them there and gone back to bed. At which point they decided it was too cold, and went and got their space heaters.
I suppose they were just going to sit and work until the financials mainframe caught fire and/or ran out of power and performed an uncontrolled shutdown on itself. I completely lost my shit and threw them out, raving and twitching like a pentacostal street preacher. They complained to their boss, who did his best to get me fired because a) I yelled at them, and b) we had to pay for more time at our remote call center in India and that's expensive.
I protested on the grounds of all the rules vis a vis our server room security, meant to cover machines that generally dealt with 6 figures a day in cash transactions, and the health and well-being of our servers. My concerns were laughed off at a local level, but I'd bbc'd one of the regional admins, who took it to the CTO who had an exorcist-style screaming fit, which resulted in a lot of people (not including me) getting fired.
I learned my lesson about battery backups. When you get the call, even if you have hours left, you've got to supervise them. It's like a death watch. Also learned a lesson about the kind of person who has server room access for the purposes of changing tapes and stuff like that.
Admin
*wrongly
Evidently some people can't use simple words correctly either... :P
Admin
Admin
Absolutely no idea where you are quoting from.. but it is much more interesting that UPS, so I'll go with it..
an Innings in baseball is roughly equivalent to... shock.. an innings in cricket. I realise this may suprise a lot of people, so I'll give you some time to recover before making any further major revelations.
Admin
There is no way you wouldn't have gotten fired for kicking the CSRs and their space heaters out of the server room. :)
Admin
WTF! That is too funny!
[image]Admin
Have a look at the source - it's where the best bits of Remy's articles are to be found :)
Admin
That story was about 100x better than TFA.
Admin
By best you mean "least worst?"
Admin
At least they were smart enough to not plug UPS #2 into itself.
"Duh. It's an uninteruptable power source. Why can't it run itself?"
Admin
Admin
Alex should write a book. Or three. Look at the first sentence, and be awed. This article comes down to "We have 2 ups's; they were wired incorrectly; it's a miracle that even in good times the servers stayed up. Now they're really old, I asked for new ones, and didn't get a response". But like an artistically drawn flower, the story unfolds into a glory of dusty old cables, flashlights without batteries, and flickering lights.
Good job Alex, keep up the good work!
Admin
Wait, if I plug my UPS into another UPS that is like double backup!
This can't fail!
Admin
Ugh, I wish this didn't hit so close to home. I inherited a server room with exactly the same setup, and I was left scratching my head as to why my old "boss" (I am a developer, he was the sysadmin... now I'm both) thought this was a good idea.
Admin
Is very easy to make such mistake! Sometime electrican person doesn't know what he is doing.
Admin
Any company with "management" stupid enough to ignore basic need for reliable power deserves to have all their servers crash and burn, unrepairably.
Admin
Ok, so, isn't TRWTF this:
"Hubert stumbled to the server room in the dark, guided only by the blinking lights of WiFi routers scattered throughout the building"......
Wait, so the power is out, but the wifi routers are on? HTF? Were they powered by 9V batteries wired in series or something?
Admin
Admin
No electrical person would do something that stupid. There are electrical codes, like the NEC (National Electrical Code) for a reason! It's only when you get ignorant computer techies who think they know electricity because they can match a plug with a wall outlet, is when you get complete clusterf**ks like that.
Admin
I may have completely missed the point or joke in this post , but an over in cricket is not roughly anything like an innings in baseball.
Admin
The British are always wrong.
Admin
Fortunately customer service isn't our forte ;)
Admin
power over Ethernet? (presumably from a switch inside the server room)