- Feature Articles
- CodeSOD
- Error'd
- Forums
-
Other Articles
- Random Article
- Other Series
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
that's a very very poor story and just states that you should learn the basics of server administration and some math before trying to make big $.
Admin
There's the old story of when Werner Von Braun's team needed to ensure that the Apollo rockets were safe and the spec was for "five nines" (99.999% success rate). He asked his top engineers if this was possible and he did get five "Neins".
Admin
This comment is 99,9% first.
Admin
Imagine my heart palpitations when salespeople were flanking me (as CTO) in big-dollar meetings, discussing "100% secure".
I don't think I shat right for a year.
Admin
Admin
We have 99.9% uptime SLA's on certain systems, but there are hot live clustered servers, auto failover, duplicated everything in DR, and so forth.
How could Gary a) give such an SLA on a single box system, and b) send a part-timer out to fix an essentially dead server without explicitly mentioning the SLA (not that Ivey really had much choice at that point)?
I hope MediumCo sues Gary's a** out of business!
Admin
Except that the rocket was the Saturn V, and that it did indeed have a 100% success rate. I believe it is the only major rocket in history to achieve this.
The Apollo capsule did have a couple of spectacular failures, however.
Admin
Poor Gary, IT just isn't the industry for him.
Admin
And yet again, another crooked snake oil salesman who knows nothing trying to pretend he's a business owner, but not spend money to ensure a proper environment. The words "cobbled together workstation running Linux" earlier in the story says it all.
I hope this jackass got sued into oblivion; he deserves it. Not that he didn't TRY by having a real server, but a server without proper support does nothing.
Admin
The Real WTF is that Ivey is spelt "Iven" one time in this story.
Oh, and who uses VNC to remotely operate a Linux server? What's wrong with good ol' SSH?
Admin
See, if your customers are dyslexic, it's much easier to sell them on a nine-fives SLA (55.5555555%! amazing!) than a five-nines SLA... easier to uphold, too.
Admin
So what "immediate resolution" did the VP of MediumCo expect, exactly? Since time travel is either not possible, or not readily available, the only resolution open here is to retroactively lower their uptime expectations
Admin
Were you expecting 99.99% or 9.9% always funny WTF worthy stories?
Admin
Admin
When business types ask for "immediate resolution", what they really mean is "I want my money back. And some extra money, besides. Hmm, how's your kidney? I want that, too."
Admin
That, plus this sentence:
"But after been fighting a long, uphill battle against some larger competitors"...
Admin
Looks like they had four-nines uptime for sure. Except when I went to school I learned that four nines are thirty-six…
Admin
This comment wins the thread, laughing hard at the five "Neins".
Admin
as someone who works on a 99.9% uptime system, with SLAs, let me tell you how it really works.
#1) it's 99.9% SCHEDULED uptime. we have regularly scheduled downtimes for maint. #2) we have at least 2 other different systems to catch downstates if the main system goes offline. By different I mean they do the basic function, but not anywhere near as in-depth as the main. #3) SLAs are graded. you miss 99.9% and hit 99.8%, it's a minor infraction. You hit 80% and you're in trouble.
Admin
There's always the classic response. "You're right, you had 2% downtime this month. Here's a 2% refund."
Admin
TRWTF is definitely Ivey's idiotic automatic patch policies on a production server. Who automatically updates a server without testing the patches first? And kernel patches? Really now. Plus, no remote console at a site that's a two hour drive away?
The boss was just being a normal, clueless boss, the IT guy screwed up.
Admin
I always love the obfuscated company names! You must be geniuses to come up with that stuff!
Admin
I thought about remote console but well - if the offsite backup == basement he might have funds. Anyway - AFAIK there as still 2.2.x servers - they just work. No need to update as You said.
Admin
Seems to me the real question is, What remedies does the contract provide for failure to meet the 99.99% uptime requirement? If it doesn't say that in that case they'll give you a month's free service or whatever, I wonder what, legally, the other party can do. Like if I buy a car that's advertised to get 35 miles per gallon, and I find that in fact I only get 34 miles per gallon, frankly I doubt I'd get anywhere in a lawsuit. Maybe I could demand my money back. I'd be surprised if a judge awarded me big bucks in a lawsuit over such a thing. Of course, judges do some crazy things, like giving people millions of dollars because they burned themselves spilling coffee in their own lap.
Admin
StruggleCo might be smarter than they look. So they guaranteed 99.99% uptime... but I don't see that they guaranteed it on a per-year basis. As long as it eventually averages out, they should be fine. :P
Admin
This does remind me of a former employer who signed a contract with a big customer to sell them one of our software packages, with a clause in the contract saying that we would make any change to the software that they requested, at any time, for no additional charge.
I pointed out to the boss that this was rather open-ended. They could demand changes that would require thousands of hours of programmer time. He replied that we were getting several hundred thousand dollars for this contract, so it was worth it. I said that if we got $300,000 but had to do $400,000 worth of work, we weren't going to make money. He looked at me like I was an idiot and asked if I REALLY thought that we should pass up a several-hundred-thousand-dollar contract. We circled around on this a few times until we both walked away convinced the other person was nuts.
That company is bankrupt now. I can't imagine why.
Admin
That's not even the best part: The SLA was violated (by over 100%) by the time Ivey drove to the data center (which was 2 hours away).
No doubt, "It's Ivey's fault," because he didn't buy (and use) a Learjet on his part time wage.
Admin
With 13 launches (according to the Wiki), how would you know the difference between a 90% success rate and 100%? Or even 80%, for that matter.
Admin
Theoretically, if you'd waited five or six years and had absolutely no downtime during this time, the 99,99% uptime percentage could be still accomplished...
Admin
If you consider Apollo 6 that nearly destroyed itself due to pogo oscillations and Apollo 13 that had a center engine cut-off on the second stage (due to similar issues) and several other issues "100% success".
Truth is, if you fly something only 13 times, you're likely to beat the odds.
Remember, the shuttle flew more successful flights up until Challenger than the Saturn V ever flew.
Don't cherry pick data. Otherwise you can have 100% uptime, until you're down.
One metric that I heard did come from the Apollo program was that every 9 doubled costs.
Want 90% reliability, it costs X. 99% costs 2x 99.9% 4x etc. And as a first metric I've found that very reasonable when it comes to datacenters and the like.
Admin
Admin
And hooray to the return of featured comments
Admin
The only way to prove a 99.999% success rate is to send up 100,000 rockets and get only one failure.
The Apollo launch record only suggests a 95% or better record. A long way from 99.999%.
Actually there were a couple serious problems that led to one engine shutdown on an early flight. Just no spectacular explosions.
Admin
Then it will be FREE because you would probably be doing all the testing for the product and people would get a good laugh of you trying to use it.
Admin
FTFY
http://www.lectlaw.com/files/cur78.htm
P.S. How bad is a third degree burn? Here's a hint: there's no such thing as a fourth degree burn.
Admin
I'm going to flex my math muscles and say it's 1/2X.
Admin
If only he had bought two servers - as we all know* 100% is just 50% twice... :(
np: Underworld - Dirty Epic (DubNoBassWithMyHeadMan)
Admin
Not even Viagra claims that much uptime.
Admin
not to be pedantic (ok ok to be pedantic) but yes there is (note: google image results are not what you want :-)
Admin
Thats what, Twitter?
Admin
If you promise 4 9's, you have to have a failover machine. You HAVE TO HAVE a failover machine. Setting up failover isn't hard, and it isn't that expensive, unless you're promising a level of throughput. Put the disks in an external enclosure, hook two machines up to the enclosure, and set the system to automatically route to the second machine if the first one stops responding.
You also need to do occasional maintenance. It was auto-patching? Seriously? And this was considered to be a good idea? You're betting your contract on a patch installing automatically and not breaking anything? And you're auto-patching the fricking KERNEL?! They'd have been much better off not patching at all.
Frankly the WTF is that these people thought they could get away with offering 4 9's in a contract and not putting any real money or effort into it. If some company NEEDS 99.99% uptime, they're going to notice when they don't get it.
Admin
Yea right. Twitter, as a service, is hanging on to 2 9's uptime by the skin of its teeth.
Admin
And 9% reliability isn't impossible. Imagine you have a process that is extremely profitable, but unreliable. If you have only, say, 7% reliability, a small increase to 9% may be very very good for the bottom line.
Admin
Technically, he's lucky he didn't guarantee 99.(999)% uptime, because that's technically 100%.
http://en.wikipedia.org/wiki/0.999...
Admin
When do we talk about global warming?
Admin
Admin
HP NonStop!
Admin
Subtlety is not lost on you.
Admin
Six nines is never down!
9x9 + 9+9 + 9/9 = 100
Admin
That initial amount was later reduced to a few tens of thousands or a few hundreds of thousands. FYI.