- Feature Articles
- CodeSOD
- Error'd
- Forums
-
Other Articles
- Random Article
- Other Series
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
I hope you meant "ludicrous, but happens all the time anyway".
Last place I worked (a very small software development company) the hard disk containing the CVS repository of all the source code crashed unrecoverably . . . and there was no backup at all. The only thing that saved them was that someone had created a fresh tree a few days earlier. Absolutely true.
Admin
I was going to say, this is at least the second time i've seen it on WTF with just a different name.
Admin
Auto-shutdown can bite you in the ass too. Let me tell you of a cavalcade of WTF's at one of my summer internships: First, the servers were in a room with water sprinklers. Big WTF right there. But it's OK because the servers would auto-shutdown if the sprinklers were going off. Certainly minimizes the damage that way.
One night they performed a mandatory test of the first alarms. Thankfully somebody had the foresight to make sure the sprinklers were turned off, to keep the whole building from getting soaked. But they forgot to disable the mechanism that tells the servers the sprinklers are on....
As we all know, the best time to do fire alarm tests like that is 3AM on a Wednesday. It's the time people are least likely to be bothered by it. Conveniently enough (and for mostly the same reasons), 3AM on a Wednesday is also the best time to backup a week worth of data.
So mid-backup the fire alarm test goes off and tells the servers that the sprinklers are on, so they shut down. The one server that has all our maintenance records (for gas pipelines, so yeah that stuff's important...not to mention federally regulated) was completely toast.
Thursday was pretty busy for the rest of the office, but interns like me got a nice break.
Admin
My we're on a real right-wing blast today aren't we.
We used to have a Technical Services manager who was nicknamed 'Yee-hah!'. This was because he had some funny habits, like walking in to the server room to borrow an extension lead, pointing to a power plug in its socket, saying 'this one isn't being used is it?', yanking it out... and zilching the main development server as a result... We only lost a week's worth of code changes, that's all
Admin
Admin
HOLY LOL!
Admin
The Side Bar WTF says 90˚ Fahrenheit. The article above says 109˚. I think someone got a bit creative there...
Admin
Fine, Urban Legend, okay, but this is supposed to be a state department? I've worked in government long enough to know that there is always, always someone in. Whether it's a three-day weekend, or the sky has gone dark from that massive incoming meteor blocking the sun, there is always someone working and that means there is always someone in IT. And I work in quite a small local authority. It's like one of those jokes that people tell as though it happened to a friend of theirs in an attempt to make it funnier. How many times do you want to tear the head off your colleagues for sending you the same damned hoax email they forwarded to you 5 years ago?
By the way, you should know that Bill Gates has teamed up with Google, AOL and NASA, and he's giving away a substantial part of his fortune. They want to track email usage so for every person you forward this to, he'll give you $10!! (my friend works in IT and this CAN be done!!1!one)
Admin
Admin
Obviously, you have never worked in academia-land - ALL Phds know more than the tech people. If you don't believe it, just ask them. While I was sysadmin on a supercomputing project, I was told (by a Phd, of course) that only a Phd could understand and work with a Unix box... too bad I didn't have any degrees, and I just made it work...
Admin
A google search for "a little more conscientious of our energy usage" turns up a number of places where an earlier version of this (which I saw on the livejournal community) is reproduced.
Notably, in the "original" story (I'm skeptical) this is a company, not a government agency, and the temperature doesn't actually get very high before someone notices and the crisis is averted. I guess that wasn't exciting enough.
Admin
OMG, they ship it along with the server? That's service for you!
If you'd ordered it from HP, they would have sent the memory along separately from the server, and it wouldn't arrive until three months later. When you rang them up to complain, they would have told you they don't sell that sort of memory in the first place.
In my experience.
Admin
I imagine his early retirement looked somewhat like this: [image]
Admin
OMG! The Daily WTF (or whatever this site wants to call itself these days) changes stories before posting them! Who would have thought?
Admin
It may be free, but it ain't excellent. I'll pick "Keep it simple stupid" any day of the week, but Nagios is way too simple and the configuration was way to messy.
Here is a paraphrased conversation I had with a British fellow who helped us set ours up (smart guy).
Me: How do we know if the Nagios server goes down and stops notifying us that servers are going down? Guy: Configure another Nagios server in another location to monitor this server. Me: Since my boss is going to ask me the same question... what if they both go down at the same time? Guy: You'll be right fucked then.
Admin
This is fake as it would not happen on most systems. hp/sun/ibm/emc/stk all have thermal shutdown on their servers and storage and have had for many years. Unless they are buying supermicros or dell there was no chance. Also the email would have gotten out if they had a redundant (n+1) AC setup as the room would have had suffeceint cooling in the room and the systems should not have overheated for at least a couple of hours. Fake but funny.
Admin
Is this The State newspaper in Columbia, SC?
Admin
So how does a redundant system help if someone turns off ALL of the units at once? if it's (n+1), and, as in the article there were three, that means n=2, which means if you turn all three off you have 0=(n-2).
Admin
You include it into a "software upgrade" or "hardware maintence" bill. Or, if you are feeling inspired, into a "office cleanning" bill. Whatever.
Of course, that will only happen if you are granted the money. Since the amount you get isn't related with the amount you need, you also spend it on ways not related to what program it goes into. It is quite simple...
Admin
Funny, but made up. We don't need more things to clutter our email.
Admin
I don't know, I was in a company a while back where I wouldn't have put it past the leadership to pull something like that. At least most of them knew enough to call us (the IT guys) before doing anything that could possibly in anyway affect the servers. And was probably only because my boss could be a real pain in the neck if you crossed him (which made him great for dealing with said leadership btw).
CAPTCHA: darwin (The should have been using Macs).
Admin
(captcha: muhahaha, how appropriate)
Admin
What a TIT!!
Just a thought - I wonder what the carbon footprint was due to the vast amounts of adrenalin and sweat the techies expended - versus the juice saved from downing the A/C?
Keep non techies away from the Kit!!
Admin
Oh this had better be a joke. If that was my server room the guy had initiated a meltdown in, he would be facing more than just early retirement.
I feel for that IT group, had sh*t like that or similar happen on too many occasions.
Admin
Admin
Blimey! Is this what liberals do in the US?
I'm glad I'm in the real world. Here we have Right wing, Left wing and the Centre.
The Right wing wouldn't have air conditioning as it would cost too much.
The Left wing would be lobbied by the unions to ensure they employed 1000 poeple to blow on the servers.
The Centre (Liberals) would get some AC and manage it properly.
At least, that's the real world in my head.. and I'm sticking to it.
Admin
This is why we don't let our programmers near Critical Hardware
Admin
Admin
I just read this in another column within the last month.
I call PLAGIARISM. The other story didn't embellish with the whole state agency thing.
It's on DIGG under "If Computers Have Cooling Fans do you need A/C in Server Rooms?"
And here's a version that reports it as "I checked email before I came in" - no mention of a state IT dept. http://worldsmostuseless.blogspot.com/2007/05/more-it-idiots.html
I'm sure I saw it somewhere else (here, on Sharktank, or Clientcopia)
In the name of integrity, I call upon Alex to remove this. From this point forward, all postings will be suspect for BS in my eyes.
Admin
Admin
I totally believe this. Depending on the jurisdiction, it can be incredibly/stupidly difficult to get rid of someone with cause. Since he can claim that he was acting (in his mind) with the best interests of the company in mind, they're not likely to recoup enough money to matter.
Also, I read "early retirement" as "he left before the investigation was complete and they could fire him".
Admin
It has nothing to do with it...that's where the humor comes in. You start at one point, and go off on a tangent that doesn't have any logical sequence from the starting point. That's why I started with "gee wonder where the heat is coming from" which is the only relevant part, and went from there. It's all in fun. Like when someone quoted my post and added, "Lighten up, Frances". That has nothing to do with what we were talking about, but it's a damn funny movie quote. As it has been suggested by other posts, this probably didn't really happen, or didn't happen the way it was presented here. So lighten up and have fun with it. The logical world will be there when you get back.
Admin
Admin
Admin
I say: fire 'em both and let God sort 'em out!
Admin
This is obviously fake.
First off, and the most obvious, would be the fact that a key is lying aroun for anyone to obtain. The same key that would allow anyone to access confidential information regarding an entire states tax collection and other commerce data. I find this hard to believe, and it this did happen someone would have a field day suing the state for inability to maintain confidentiality. Nothing any state wants to deal with, and it's employees would not publicly post their own failure to maintain. Not to mention the confidentiality agreements, either written or implied, regarding a states computer network. Can we say terminiation?
A 109 degree computer room? Must have been a small assortment of equipment. Most computer equipment can take temps much higher than that, although improperly vented racks can get hotter inside. Call it poor planning on the data center design for not putting heat sensors in with their tertiary A/C, and network/system admin's fault for not having a simple alerting system in place. Hell a majority of servers come with some type of temp/system monitoring software, idiots if they weren't using it.
Got what they deserved for building a half-baked data center for an entire states tax collection purposes. Had the story been factual, you can guess who paid for the restoration.....the taxpayers.
Admin
You could at least write a script which runs on each machine and pings all the other machines every minute...sends an email/text message/whatever when a server disappears.
Put at least one server on a UPS (they're not that expensive!) so that if the power supply fails all of them simultaneously, there is still one alive to send the SOS. If you can't afford a UPS, then use a laptop.
But honestly, you could afford a generator, but you couldn't afford to page somebody?
Admin
Haha, what did you expect him to say?
Guy: Well, actually, angels watch over the Nagios server, so if it does go down, not only will they magically resurrect it, but you'll also be blessed with consciousness on your death bed. You: So I've got that going for me.
Admin
Bcos the Bible sez we hafta take the Earth and rape it, it's ours.
You never listen to Ann Coulter?
You'll have to agree that a government agency is a much better choice than a privately-owned company if you're going to entertain conservatives.
And you bet they didn't open a hatch in the ceiling.
Captcha: burned (sorry, it was just too funny)
Admin
Hey! There's no reason why a faith-based server monitoring program wouldn't work.
Admin
Things like this do happen. Looking on Snopes, there's no mention, and the only hits on most of the terms point back to here! Variation on a theme perhaps.
By the way, a room full of servers like that with 3 running A/C doesn't go only to 109F. And even if it was that low a temperature in the room, a computer can run at 109F just fine.
For the other subject, the chances are somebody who is a conversationist to that degree is of a more liberal political leaning than not, and it has nothing to do with intelligence. Those are generalizations of course.
Admin
The whole green movement feels kinda hippyish, and they're also stereotyped as being a bit strident and narrowly focused, so there ya go.
Have both servers look at each other. The probability of both going down without a common cause is miniscule, and something that takes them both out, you're likely to notice anyway, like a major power outage (of course, the mail wouldn't work, so it doesn't matter).
Admin
I actually worked in a company where the owner tried to save money by shutting down AC over weekend and every night in the server room. Setting aside the fact that it was stupid, inefficient and risky...Every morning I came to work in the heat of about 95 F room where servers were running just fine. Actually, it is very hard to find statistically meaningful data about the failure rates vs. temperature. I tried. Some general ideas - sure. But nothing to build the business case on. I vote fake.
Admin
So Cho was just a pissed off environmentalist?
Admin
Now, I know that in politics talking often means hot air, but even so...
Pete
Admin
http://www.helenair.com/articles/2006/05/24/montana/a08052406_01.txt
Admin
This sounds pretty fake. I was a senior Unix admin at a very large telco manufacturer with a very large server room. We had about 500 servers in there, everything from a Sun Enterprise 250 to E10k and Sunfire 6800, etc etc. Lots of EMC disk cabinets, NetApps, etc.... The weird thing was that the AC was on a separate power feed that was unreliable (the actual electrical company had trouble keeping the feed into the complex up). So we would lose power to AC with the servers up and running- and when that happened it got HOT - easily 100. The only time we lost disk (or any hardware) was inside the disk cabinets after being used 24hrs a day from around the world - and it was a grand total of 5 disks we lost. I find it hard to believe a server room this small could have such a huge hardware failure, even if the AC was out for a week.
Admin
It's not just money. There are security issues too. Connecting your box to the outside world well enough to send an outgoing page can easily get your box owned, or worse.
Admin
I stood 20 feet away from an operating AS400 once.
The wind from the exhaust fans was enough to make it inconvenient to read a pamphlet I was holding at the time...
Admin
When I read this story and think of "retirement" I also immediately think of "Blade Runner."