- Feature Articles
- CodeSOD
- Error'd
-
Forums
-
Other Articles
- Random Article
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
I get the idea of a "temp" server. It is quite useful to have a place to use things for evaluation or in a restricted eviorment. But even in the example you listed the "box can be pulled at anytime" policy is counter productive because you have just finished installing BizTalk (which takes quite a while if I remember correctly) and the machine gets wiped before you get a chance to evaluate it. A whole day's work is lost that could have been saved by a "Is anyone using server X" email.
But like someone said earlier. We're all making assumptions as to what the definition of "temp" is in their enviorment. The machine was up for over a year. I wonder if pulling down a "temp" server is a common occurance.
Admin
Even though I wouldn't count those two as code-related (see below) I'll forgive you since you replied ;-P.
Abstract Candidate I wouldn't classify as code-related because the WTF isn't in the code really.... The "programmer" couldn't even code to begin with. Also, the "code" doesn't even compile lol.
Poor Mr. O'Hare only HALF counts. It qualifies the below criteria, but c'mon... I want more than one line!
I'm talking things like When (n<0) won't do. You know. The kind of wtfs that produce code that executes properly, and does stuff, but does them in the worst possible way (or damn near close to it). You know, the kind of stuff where you know the guy/gal coding the stuff is at least partially intelligent (they do come up with very, uh.... very "creative" solutions) but he/she just doesn't realize how retarded the stuff he/she is doing is because he/she fails to understand some simple concept or something of the like.
It sucks that you haven't received any new bad patterns yet, I'm sure there are plenty more. Also, why not just re-do some wtfs that may repeat the same pattern, but in a new way?
Admin
Everyone assumes that was what happened. All they really did was use SourceSafe.
Admin
Gotta love this PM's professionalism....
1. Tool he must have isn't specified in his contract so he pirates it.
2. Doesn't engage any kind of disaster-recovery plan for project management.
3. When faced with the inevitable catastrophic failure he simply abandons the project
Gotta wonder how he'll explain away the year-long gap in his resume to his next project....
Some people are blaming IT for the server wipe, but that's like blaming the hard drive for failing. The root WTF here is not the method that lost the project documents, but the fact that this large project was managed so poorly.... Yes, IT could have been more proactive, but really it's the PM's job to ensure proper project resources and dedicated hardware/software/support....
-me
3466 Firebug errors writing this message!
Captcha: what captcha, I'm registered....
Admin
In a creepy way, it reminds me of the "Monorail" episode of The Simpsons.
Admin
The essential problem here is that they just assumed the machine wasn't being used before wiping it. Just because things SHOULD be done a certain way doesn't mean they actually are being done that way. In the real world, people don't always use boxes for their intended purpose, because they're dumb. Having dumb people working at a company is already bad, but wiping out a year's worth of their work just makes matters worse. I'm not saying Network Ops is the only one in the wrong here, but they could have ultimately saved the organization much frustration and lost time if they had acted properly as the last line of defense against data loss.
Admin
I used to manage a single server that was used exclusively for experimental software (during the IE/Netscape release-of-the-hour browser wars). I would be forever reinstalling software on it (Orbix/Visibroker/NES/... - all alpha releases). About every 7 or 8 days, I'd need to reinstall windows because there was just that much crap and no way to undo it. Every couple of weeks I'd need to wipe the whole disk and start clean.
The rule was that you could access the server to try the new software, but never put anything on it.
Every once in a while, someone would put something on there without telling me. One such person in particular used it as a backup drive - without telling anyone. They never noticed that their backups were being obliterated because they never looked at them. I found out about it 6 months later when they went to retrieve something from their backup files, couldn't find it and started to panic. They went to my boss, and her boss, and complained - loudly - that I trashed their backups. I got called in to explain. My response:
"What do you expect when you use a dev server named: DoNotUse for backup?"
Admin
It would however be a learning experience for more readers than would care to admit it ... ;-)
Admin
My last company played it safe through the use of a post-it note (for system name, machine name, date) and a queue of yanked hard drives. If no one complained before it got to the front of the queue, it was considered safe to wipe
Admin
My distinguished colleague said:
Frankly, there's no room for fairness for this individual. He took a DEMO product and fashioned it into a production tool. It's like working the high-wire without a net. Just plain careless. I appreciate that he didn't get the tool he wanted but sometimes you must work within the bounds of a budget.
"An ounce of prevention is worth a pound of cure."
Admin
Too stingy to purchase the right software?
Well, you get what you pay for, eh?
Admin
My friend and peer said,
One co-worker of mine from years ago found out how to store files under his mail account on our XENIX mail server. When the server crashed we had a backup, but didn't appreciate restoring 300Mb (long ago) of his porn.
Hmmm.
Admin
I agree. No self-respecting IT person would do such a thing!!
Admin
Didn't anyone think to put the printouts on a table and photograph them?
Admin
Sometimes it's not a matter of respecting yourself. It's a matter of nursing a slow-burning hatred for everyone else.
Admin
I would bet that IT sent out a notice that server xxx was going to be repurposed, but that no one in the project group knew that they were on server xxx and what was going to happen, so ignored the email.
Admin
Depending on the company, this might be even worse. By sending such an email, you confirm that you suspect that important stuff could be on the server (so forget "plausible deniablity") and still there might be someone on vacation who has stored something of importance on that server.
Well, a lot of people could have ultimately saved the organization much frustration and lost time... the network ops are IMO the last to blame. At least, they followed the rules.
Admin
It wasn't a random box. It was a very specific box; one that was to be used for temporary, "Danger - this machine can be wiped at any time" purposes.
Admin
The onsite job I'm at currently has all kinds of WTF's. Among them is the servers are the all-in-one variety. The development server host SQL Server with about 15 development databases, IIS development web sites, file storage, VSS, etc. Likewise for the Test server. The Production server is similar with database, IIS, and code programs all-in-one. (At least they try to back everything up, not sure of the succes rate though.)
Thank goodness for USB flash drives! My work and all critical related files are backed up to it almost every evening. If they lose the server, I have my backups.
Regarding the school projects and backups, for several years I had backups of my work on 8-track tapes, 5 1/4 floppies, plus paper copies.
Maybe it is a character flaw, but I have never trusted a single copy in one place of anything that has any importance.
Admin
Under which language does "Perforce" mean "Subversion"? ;-)
Seriously, SVN has got to be my favorite source control program, especially for my private projects. It works extremely well under Windoze.
Admin
It is impossible to waste parts of your life because of someone else. It is possible to blame someone, yes, but it is impossible for it to be the other person's fault.
Admin
Having dumb people working at a company is already bad, but wiping out a year's worth of their work just makes matters worse
I dunno, their code was probably full of WTFs anyway ...
As for the thread starter, that's a fairly spectacular corporate cock-up. For me, the biggest WTF is using a demo server for production processes without telling the network admins. The admins were a bit rash to wipe without warning, but that's what happens to stuff on dev-temp servers.
By the way, I know it's been done before, but the editor on these forums really is the Real WTF.
Admin
wtf, that's painful reading.... [image]
Admin
That would make a great sig.
Admin
So if you're using the sandbox to check out BizTalk, you wouldn't be annoyed if someone took your sandbox and wiped it overnight? Even if there's no data or state on the machine, its going to set you back time to get the software you were testing set up again (and to get new hardware now that the machine you were using was repurposed).
Admin
I'm thinking about submitting it for our "Vision Statement" contest.
Admin
I think the likelyhood that an executive would blame you for sending out a notice and then deleting important data on a "temp" server because you "suspected" that there was important data there just because you sent out a warning notice is far less likely than getting blamed for not following common sense precautions and by not sending out a warning message. Maybe when playing golf you dont' yell four because that would mean you thought you were hitting a ball purposefully at someone. Or you don't wear seatbelts because that would be admitting that you're a bad driver. Just because someone may possibly act stupid is no reason for you to act stupid.
And we don't know what the full policy at the company was or was not regarding the temp server. But even if I have the choice of following a company policy to the letter of the law and not using common sense, or to go above and beyond the company policy and use common sense, I will use common sense every time if there is even a remote possibility that not doing so could cost the company millions. And I've worked at many companies that had policies for dev temp machines that required notice to all possible parties that could be using the server, i.e. all users who have access to the server over the network. And some companies, even required changes to dev temp computers to come be fore the Change Control Board for approval.
Sure it's possible that there company policy expressly forbid sending out notice that a dev "temp" machine would be wiped. But there is nowhere close to definitive proof of this, and I've yet to hear of a place with a policy like that.
Admin
Holy shit.
I'm sorry, but... what?You're saying that you weren't graded until the very end of the four years? Worst-case scenario, it should have wiped everyone's work for the semester, but the machine should have been backed up and you should have had copies of all your work on your personal machine anyway.
Admin
Where I work, pretty much every server is backed up - unless it's in a "lab" such as DR, SOE Testing, etc. Even those - in the labs - are, in effect, backed up, because they are restores from other environments.
Yes, backup is expensive. Could be $50k to backup 2TB, for example. One salary.
There's no such thing as too expensive to back up, no excuse for anyone - admin teams or individuals - not to back up.
On projects that I work on (all development - they then get handed to support, and we're onto the next project), the Development server is the most important. Lose a day's work on that, and it's cost you 10 days worth of highly paid developers. Lose production for a day, and you fail back to your log shipping DR box. Costs about 1/2 a day of a support person running around and making sure the business understand that they've temporariliy up to an hours transactions - and they'll get them back soon, so just sit tight. About 1/2 a day to coordinate the fail back and execute when required.
I'm not saying prod backups aren't important, but when evaluated in terms of lost productivity for an entire business, dev backups are as least as valuable - I've had no trouble convincing bean counters that this is the case for 15 years.
Admin
Everyone comes up with this "server will be formatted in a week" but forgets about the dev's request for a terminal server. If I requested a terminal server and I got the reply "you'll have it in a week", I'd be really mad and complain loudly about how crappy this policy is. You all say that wiping a temp machine without notice is counterproductive, but how about having to wait for a simple terminal server for a week?
Yeah, you're right, the IT should have a pool of unallocated servers but the someone asks for a new temp server and another one and the pool is empty. And we all know how quick the process of buying a new one is at a company of this size.
So what will IT do? Wipe the oldest temp server allocated. Don't forget that it had been allocated as temp for over a year . Now having temp data for over a year is obviously nonsense, isn't it? Again, I can only assume that the general level of professionalism of the company's dev team used to exceed that by lightyears. Before the super-PM arrived, that is.
Admin
Admin
If that's the case, I'd be tempted to say that The Real WTF (TM) is that this machine was plugged in and running for a year and half without being used for anything. If it's a machine that could be taken away and wiped clean at any time without notice, nobody (in their right mind) would ever use it. If the machine is never used, why even have it running?
Captcha: hotdog. Just what I planned on making for dinner tonight...
Admin
Really, the blame rests with the PM. Yes, IT probably could have handled the situation differently but if you read the original post, R.B. repeatedly told management and everyone that storing the data on a temp server was a bad idea for this very reason. A very minor IT WTF I think but a HUGE "Expert" Project Manager WTF.
Captcha = clueless... like the "Expert" Project Manager
Admin
Three sides to this WTF coin, the front face ( Development ), the back face ( IT ), and the outer edge ( Executive Leadership ). Each is equally complicit in it's failure of responsibility.
In a flip of the coin to assign blame, odds are - the coin won't come to rest on it's outter edge.
Admin
I think the real WTF is that with such a huge budget for the project they couldn't afford a few thousand $'s for a server... I mean seriously... you can get a kick ass server for less then $5k nowadays, so I think it's really a management WTF for not letting them get another server (although I wonder what the cost of the software was, and I guess they didn't want to pirate the software by setting it up on another server, and this one was supposed to be only for test purposes...)
Admin
reminds me of when a guy we were working with, a supposed oracle guru, was managing our production oracle servers... one of the juniors accidentally did an update with a missing Where clause and updated all the rows in a very large table, then did a commit.... no big deal, had happened before, and we would normally just roll back to the previous day... well thats what we thought we could do until we found out that the daily backups didn't happen because the oracle guru had told the network team they didn't need to worry about backing these databases up, because he'd written a script that would back them up quicker than their other software....
His script di something, i don't know what, but it basically would copy the information down to his local machine... which was good and all, except his local machine was a laptop.... that he took home each night.... there had been no backups done for around 6 months
Admin
Grandfather from The 5th Element:
You are fired!
This is a good WTF. Nice to see that I wasn't the only who had to put up with similar stupidity.
Admin
The important phrase is "in one place." We had some workers that resided in a portable office building, and had been doing research for about two years. They did regular backups of all their programs and data, which were then stored in a file cabinet in their office. When a forest fire burnt their portable office building, it destroyed their computers and their backups. Truly two years work "up in smoke."
Admin
You know, I thought about that for about fifteen seconds before posting before settling on Perforce. I like Subversion, too, but if I were in a managerial position selecting a mission-critical SCM system, I'd select Perforce.
Once Subversion earns a couple more years of a trouble-free reputation, I'd probably be much more comfortable selecting it.
Admin
Totally - that's what got me into this site :-) Real-world antipatterns are very instructive, and I'm sure plenty of newbie programmers have learned quite a bit from these pages.
That said, I like Symbolic Installation (the Wingdings story) - possibly my favorite "management" (i.e. non-code) WTF.
Admin
Alex, thanks for not totally concentrating on code specific issues. Coding issues are fun because they are fun to solve. However, The whole picture is important. Workplace, collaboration, architecture, security, support, planning, marketing, and management issues complete the picture.
This WTF re-affirms that code is really a small cog in the big machine... That at any point in the big picture, someone can drop the ball and loose the game for everyone... That our best efforts are vulnerable to any idiot with a keyboard and credentials.
Yet...in blind defiance of better judgment, we relish the rare, somehow workable, profoundly stupid... code example. We need a good dose of really bad code.
Admin
Suppose that netops' usually generous bonuses were dependent on the company making their numbers -- which this fiasco prevents. Guess they can console themselves with the fact that 100% of the blame is on the PM...
Sometimes a little proactivity pays off.
Admin
Yeah man, don't blame me for your mistakes! PHP is gay anyway.
Yeah, and subversion doesn't have a cool walking P4 dude as a busy cursor. Lame.
And for GoatCheez, just to make <FONT color=#ffffff>troll</FONT> you <FONT color=#ffffff>troll</FONT> happy <FONT color=#ffffff>troll</FONT>......
Admin
Oh. My. God.
I would have demanded the school also find that man/woman and assassinate them for me, so that I wouldn't have to go to prison to see them die.
That pisses me off just hearing it, and it doesn't effect me in any way!
Admin
Wrong... the blame falls on all participants. All shared in a responsibilty that they should have known better. It's a best practice WTF.
Admin
The code is written in Enterprise Micro Funge which, while it has no published specs, is a fungoid.
The code in question was anonamised. The original code was in Funge-98.
Admin
What reason would a company have to ever make such a machine accessable to users? It's a business right? Even sandbox work is work, and costs time/money. I can't think of any situation where a machine can be safely nuked without everyone with access to it being notified. That's why you see scratch spaces that say "your file may be deleted anytime after X days or even X hours. A scratch server where your files may be deleted 30 seconds after creation isn't useful, not matter how trivial it is to recopy the data.
If there was no documentation of something running on the server, then no one should have had access to it. That is a first order sysadmin/operations WTF.
Admin
I would have fought through my teeth for the peice of paper at the end. Worst case scenario, I would have ask for a refund for the final year project, and repeat it the next year.
Maybe Australian Universities/Colleges are different, but our courses were all very disconnected from each other. i.e Our java units between semesters had nothing to do with each other. To suggest that someone else's stuff up means you have to start again is down right madness!
Admin
Certainly not the best move using "unsanctioned" software for everything but I have had a systems department wipe a development box without asking.
Definitly negative marks for the EPM, but I'd be knocking on Systems door to explain this one.
Admin
meant to say never had a box wiped... time for bed...
hmm... funny... the captcha is "bedtime"... I guess the universe really means it this time.