The Daily WTF: Curious Perversions in Information Technology

chubertdev · 2013-04-22 Reply Admin

I'm surprised that this one didn't have a "John sat his desk being paid to do nothing" or "John then fired up Word" ending.

Some Damn Yank · 2013-04-22 Reply Admin

notchulance:
This is why we need to bring back critical thinking in school. We're becoming a civilization of mindless drones who can't think outside the box.
^ This.

I find it hilarious that the quote and your reply reside inside a box.

2013-04-22 Reply Admin

In a perfect world, employees would be rewarded for creativity and taking initiative.

In practice, even in a perfect world, there are two big problems with this.

Often when there is more than one way to do something, it is better if everybody does it the same way. Like, at my company we use SVN for our change control system. Suppose tomorrow I discovered some other CCS that I think is better than SVN. So I set up my own repository using this new product and I put all my code there. What happens when someone else needs to work on my code? They need to not only install the software, but also learn how to use it. Suppose another employee decides there's some other product he likes better still. Before long we have 20 repositories and nobody knows what's where, and nobody really knows how to use any of them effectively.

Sometimes, often, doing something the same way everybody else does it is better than doing it your own way, even if you are convinced that "your way" is better. Even if your way is objectively better by any reasonable standard.

Of course this can go wrong if the organization gets stuck in the mud and refuses to consider new and better ideas. But sometimes a better idea isn't ENOUGH better to be worth the trouble.

In practice, many employees are not sufficiently skilled or talented to be trusted to make certain categories of decisions. I've known many, many programmers who are competent to write code, but who know nothing about database design, or who could tell you the relative merits of one language versus another, etc. So what really happens in most organizations I've worked for is that someone has to prove himself before he is given the authority to make certain kinds of decisions.

Of course that process is far from perfect. Sometimes people are given authority because they are good talkers even though they know nothing, and they screw things up right and left. Many people who are quite competent are never given the authority they should have and become frustrated that their ideas are always ignored. Etc.

So I don't think the problem is that people generally follow the rules. The problem is when the higher ups are unwilling to consider the possibility that the rules are poorly conceived or out-of-date, or that there are times when there should be exceptions.

2013-04-22 Reply Admin

QJo:
TRWTF:
A horrible dread crept over John’s skin. “I must have left in some flags, or some settings- it’s pulling data from PerfTest..."

The exact same build that is deployed in Test should be able to be used without changes in Production. If you need to make amendments to the code base before you perform the Production deployment you're doing it wrong.

In theory... Sometimes you need to deploy a data-source definition where the name of the database-instance changes between environments. Or your Log4J needs to have debug-logging enabled in dev, but never in production. Or your a new database-table needs to use different tablespaces.

You can try to automate it as much as possible by using templates for configuration-files and some clever scripting that creates the actual files as part of the installation, but sometimes stuff goes past unnoticed.

2013-04-22 Reply Admin

And by the way:

If you follow the rules, and things go badly, it's not your fault, it's the fault of the person who made up the rules.

If you break the rules, and things go badly, then it's your fault.

If you break the rules, and things go well, you may still get in trouble for breaking the rules.

So from the employee's point of view, the safe thing to do is always to follow the rules.

                | follow rules           | break rules |
----------------+------------------------+-------------+
turns out well  | you win             +5 | toss-up  -1 |
turns out badly | blame somebody else +1 | you lose -5 |
----------------+------------------------+-------------+

Clearly the best strategy from a game theory point of view is to follow the rules.

da Doctah · 2013-04-22 Reply Admin

Craig:
This forum needs a "Like" button.

Never mind applauding. If you like what you see, send Bitcoins.

2013-04-22 Reply Admin

From the article:

Orders, orders everywhere. Tens of thousands of orders flooded into the production system. A glance at the log files explained their origins: the performance testing environment.

I'm not sure if this has ever been covered here, but the above statement basically describes what happened to Knight Capital last year. The orders originated from a test system and thus they had no risk limits in place. They were sent to the New York Stock Exchange and they couldn't just be cancelled. Knight had to trade out of them. They lost over $400 million in about 40 minutes, mainly because nobody could figure out how to shut it off. It ranks up there with Therac-25 as one of the biggest software WTFs ever.

2013-04-22 Reply Admin

andytech:
QJo:
TRWTF:
The exact same build that is deployed in Test should be able to be used without changes in Production. If you need to make amendments to the code base before you perform the Production deployment you're doing it wrong.

This.

Also, production should be isolated from test environments (firewalls, different authentication credentials, etc) so it is simply not possible for them to communicate inadvertently.

Came here to say these things.

Glad I'm not alone.

2013-04-22 Reply Admin

QJo:
TRWTF:
A horrible dread crept over John’s skin. “I must have left in some flags, or some settings- it’s pulling data from PerfTest..."

The exact same build that is deployed in Test should be able to be used without changes in Production. If you need to make amendments to the code base before you perform the Production deployment you're doing it wrong.

Or the install team copied the web.config when they shouldn't have...etc., etc., etc.

There's reasons these things happen that have nothing to do with the code...

Matt Westwood · 2013-04-22 Reply Admin

PRMan:
QJo:
TRWTF:
A horrible dread crept over John’s skin. “I must have left in some flags, or some settings- it’s pulling data from PerfTest..."

The exact same build that is deployed in Test should be able to be used without changes in Production. If you need to make amendments to the code base before you perform the Production deployment you're doing it wrong.

Or the install team copied the web.config when they shouldn't have...etc., etc., etc.

There's reasons these things happen that have nothing to do with the code...

Configuration management fail, then, whatever. It's still bollocks. These are things which should never happen. If it does, that means someone has fucked up big time. No excuses for shit like this.

PedanticCurmudgeon · 2013-04-22 Reply Admin

Coyne:
wall of text

tl;dr: TRWTF is people making decisions but not having to deal with the consequences.

2013-04-22 Reply Admin

jay:
And by the way:
If you follow the rules, and things go badly, it's not your fault, it's the fault of the person who made up the rules.

If you break the rules, and things go badly, then it's your fault.

If you break the rules, and things go well, you may still get in trouble for breaking the rules.

So from the employee's point of view, the safe thing to do is always to follow the rules.
                | follow rules           | break rules |
----------------+------------------------+-------------+
turns out well  | you win             +5 | toss-up  -1 |
turns out badly | blame somebody else +1 | you lose -5 |
----------------+------------------------+-------------+
Clearly the best strategy from a game theory point of view is to follow the rules.

Actually, the best strategy is to try and become the person who makes the rules.

2013-04-22 Reply Admin

I want to work at a place where cake falls from the ceiling!

2013-04-22 Reply Admin

Nagesh:
miko:
Regarding the fact that he typed "like the hacker in a bad movie" - what happened to the president's daughter in that movie?

i have news. the president's daughter is alive and well. <img src="http://a57.foxnews.com/global.fncstatic.com/static/managed/img/World/660/371/rosichavez_instagram.jpg?ve=1"" border="0" /> here is her latest photo.

Where's the wooden table backdrop? This is not a high quality image.

2013-04-22 Reply Admin

jay:
And by the way:
If you follow the rules, and things go badly, it's not your fault, it's the fault of the person who made up the rules.

If you break the rules, and things go badly, then it's your fault.

If you break the rules, and things go well, you may still get in trouble for breaking the rules.

So from the employee's point of view, the safe thing to do is always to follow the rules.
                | follow rules           | break rules |
----------------+------------------------+-------------+
turns out well  | you win             +5 | toss-up  -1 |
turns out badly | blame somebody else +1 | you lose -5 |
----------------+------------------------+-------------+
Clearly the best strategy from a game theory point of view is to follow the rules.

Ha, only if.... When I worked at Fortune 500 IT, we have a team of app owners who refuses to follow any of the IT procedures we're told we must follow. Oh, they love to slam last minute changes on Friday afternoons at 4:50, thinking that all the IT "process people" have gone home, so they can try to bully you into breaking procedure. If that fails, they put our names to the C-level execs and threaten termination. So it's more like...

                | follow rules          | break rules |
----------------+-----------------------+-------------+
turns out well  | you did your job      | you risk job|
turns out badly | you risk job          | you lose job|
----------------+-----------------------+-------------+

That's why I don't work there anymore.

Mithrandir · 2013-04-22 Reply Admin

(mistaken comment in here that I can't delete)

Mithrandir · 2013-04-22 Reply Admin

jay:
At least here in the US, just look at our school curricula. Is it filled with "corporations are wonderful" propaganda? No, just the opposite: it's mostly trashing big business as evil and teaching students why socialism is a great and worthy cause. The other things that my kids' schools continually teach are: 1. racism is the root of all evil; 2. tobacco is evil; 3. guns are evil; 4. environmentalism is an unqualified good; 5. homosexuality is good, and any questioning or discussion of this point is unacceptable. Math, science, and history come in at about #42, #67, and #83 on their priority list.

They used to say that about the school curriculum when I was in school. It wasn't true then, either.

2013-04-22 Reply Admin

Where's the WTF? Isn't this "all systems normal" in most organizations?

2013-04-22 Reply Admin

Cujo:
BTDT. I worked in an environment where the "Change Control" team had no idea what they were approving but relied on the buddy system. The simplest changes, in this case dropping an index which changed the response time from 20,000 seconds to 1,200 seconds on a batch job is still languishing from 2006. I still get mails about rebuilding the index from my old job.
OTOH, the most dangerous changes went right through. At one point I had over 16 months of backed up change requests before I gave up and moved elsewhere.

I have never figured out how people who know nothing about a system, refused to read the documentation the insisted upon and cannot make decisions are the ones who decide what is important. Captcha: consequat <- Should be a real word.

For those who haven't realised it yet, IT (everywhere) is messed up, and it's as much the techos fault as the non-technical...

One of the biggest problems in IT is that people in positions with "manager" in the title (ie not just actual managers, but also release managers, change managers, incident managers etc) are very often non technical (or barely technical). Of course, we propeller heads quickly identify this issue and blame piss-poor non-technical people - but the problem is partially us. None of us want those boring positions. None of us want to be pen pushers, we thrive too much on the thrill of bug-hunting and the joy of seeing something we've written actually work. <cliche>It's a catch 22 and there is no silver bullet</cliche>. It is f'ed until people in management like roles actually understand what their role is and what they're supposed to be doing. At the same time, many of those who actually understand the role of these different positions (and understand that process has to be flexible to accomodate quick deployment of simple but critical changes as much as more considered deployment of complex, potentially high impact changes) are usually the people at the coalface making the change. I have worked for and with several TLAs (Three Letter Acronymns) who between dodging blame and saving money seem to have a layer of managers who's ability to handball would seem them star in any Aussie Rules team. These are the people who come to Post Incident Review meetings ready to fire with answers like: "I'm afraid I don't have that information, but I'll be sure to pass on your concerns to the technicians" or "I haven't got that detail on hand, but I'm expecting a summary of some totally unrelated data from our technical guys".

Perhaps the problem is the concpet of blame. With work often contracted out to the lowest bidder any issue becomes a game of dodgeball rather than any attempt to remedy (and God-forbid prevent) any issues identified. Rather than the idea of actually supporting a clients systems, the focus seems to be on avoiding any SLA breaches. Anyway, I'm getting too ranty....

steps off soapbox

spamcourt · 2013-04-22 Reply Admin

DB:
Nagesh:
miko:
Regarding the fact that he typed "like the hacker in a bad movie" - what happened to the president's daughter in that movie?

i have news. the president's daughter is alive and well. <img src="http://a57.foxnews.com/global.fncstatic.com/static/managed/img/World/660/371/rosichavez_instagram.jpg?ve=1"" border="0" /> here is her latest photo.

Where's the wooden table backdrop? This is not a high quality image.

Can't you see the photo has a brown tint? That's the latest trend in computing, wooden tables are a thing of the past. Think of it as a "virtual wooden table".

2013-04-22 Reply Admin

TRWTF: QA, UAT, and performance tests passed... He checked in the code. Really? Finish all of your 'testing' processes, THEN check in the code?

Or maybe that should read "He deployed the release to production", in which case TRWTF is why the developer has any role to play at that stage anyway - these guys are big enough to have a separate QA section but the developers are doing the production releases?

2013-04-22 Reply Admin

TRWTF is that anyone would blame the code in the first place. By the sounds of things they have a whole pile of QA and approvals that must take place before something is put into production, so I would assume code review is part of this.

If someone "must" be blamed, then its a senior person's fault for not ensuring the process was followed or for not ensuring that the process can actually pick up errors.

2013-04-22 Reply Admin

All that, and no one mentions the dumb-as-rocks sysadmin who says "No need to roll back, we'll just tweak the settings?"

Seriously? If one of my applications went tango-uniform, and the sysadmin said that to me, I'd knock him in to next week and kick his ass on Tuesday. There's a reason my deployment instructions include rollback instructions: so that if the deployment is @&#^ed for a cocked hat, you can ROLL IT BACK.

That's TRWTF, right there. CM mistakes happen, young turk developers screw up, managers push too hard to make an unrealistic schedule ("We have to do it quick, we don't have time to do it right.") But when you're in a bad state, you back out of it. You can't "tweak" your way out of a train wreck.

2013-04-23 Reply Admin

Aaron:
Where's the WTF? Isn't this "all systems normal" in most organizations?

systems normal, all WTFed up

Matt Westwood · 2013-04-23 Reply Admin

Mithrandir:
jay:
At least here in the US, just look at our school curricula. Is it filled with "corporations are wonderful" propaganda? No, just the opposite: it's mostly trashing big business as evil and teaching students why socialism is a great and worthy cause. The other things that my kids' schools continually teach are: 1. racism is the root of all evil; 2. tobacco is evil; 3. guns are evil; 4. environmentalism is an unqualified good; 5. homosexuality is good, and any questioning or discussion of this point is unacceptable. Math, science, and history come in at about #42, #67, and #83 on their priority list.

They used to say that about the school curriculum when I was in school. It wasn't true then, either.

+1. Take a bow, sir, for saying something that's gone unsaid for too long.

2013-04-23 Reply Admin

TRWTF is they don't have an "offshore" guy from India, who could be blamed for their screwup.

2013-04-23 Reply Admin

jay:
... At least here in the US, just look at our school curricula. Is it filled with "corporations are wonderful" propaganda? No, just the opposite: it's mostly trashing big business as evil and teaching students why socialism is a great and worthy cause. The other things that my kids' schools continually teach are: 1. racism is the root of all evil; 2. tobacco is evil; 3. guns are evil; 4. environmentalism is an unqualified good; 5. homosexuality is good, and any questioning or discussion of this point is unacceptable. Math, science, and history come in at about #42, #67, and #83 on their priority list.

What? Seriously? When did the BBC take over US education?

2013-04-23 Reply Admin

jay:
The other things that my kids' schools continually teach are: 1. racism is the root of all evil; 2. tobacco is evil; [..] 4. environmentalism is an unqualified good; 5. homosexuality is good, and any questioning or discussion of this point is unacceptable.

But those are all true.

ClassWarfare · 2013-04-23 Reply Admin

ObiWayneKenobi:
This is why we need to bring back critical thinking in school. We're becoming a civilization of mindless drones who can't think outside the box.

See Dumbing Us Down, a treatise by a New York teacher of the year on how this is exactly what's happening.

If you really want to understand why the US is falling behind its historical pace, it's because our school systems no longer teach independence, critical thought, or anything that might be useful outside of working a 9-5 and staying firmly entrenched in the middle class.

2013-04-23 Reply Admin

Ironside:
"This is why we need to bring back critical thinking in school. We're becoming a civilization of mindless drones who can't think outside the box."
That's a thought violation! get back in line citizen or you will be reported.

Can't have your own guns -> Can't have your own children -> Can't have your own brains.

Now citizen, get in this giant test tube thing full of breathable liquid so we can plug your brain into our parallel distributed processor, and simulate more war games.

2013-04-23 Reply Admin

ClassWarfare:
ObiWayneKenobi:
This is why we need to bring back critical thinking in school. We're becoming a civilization of mindless drones who can't think outside the box.

See Dumbing Us Down, a treatise by a New York teacher of the year on how this is exactly what's happening.

If you really want to understand why the US is falling behind its historical pace, it's because our school systems no longer teach independence, critical thought, or anything that might be useful outside of working a 9-5 and staying firmly entrenched in the middle class.

Because they need just enough middle class to fund welfare, but not too much because the poor need to outweigh them in votes.

The rich, they're 1%, so their votes don't count. We give them just enough tax breaks so they don't pay off lobbies.

Rich tax income is fixed, poor tax income is negative. The middle class is the only place where we can continually raise taxes.

lanmind · 2013-04-23 Reply Admin

Nagesh:
miko:
Regarding the fact that he typed "like the hacker in a bad movie" - what happened to the president's daughter in that movie?

i have news. the president's daughter is alive and well. <img src="http://a57.foxnews.com/global.fncstatic.com/static/managed/img/World/660/371/rosichavez_instagram.jpg?ve=1"" border="0" /> here is her latest photo.

Holy shit, she's got like... $40 or $50 there! Now she can go out and get whore ink on her lower back!

2013-04-23 Reply Admin

DB:
Nagesh:
miko:
Regarding the fact that he typed "like the hacker in a bad movie" - what happened to the president's daughter in that movie?

i have news. the president's daughter is alive and well. <img src="http://a57.foxnews.com/global.fncstatic.com/static/managed/img/World/660/371/rosichavez_instagram.jpg?ve=1"" border="0" /> here is her latest photo.

Where's the wooden table backdrop? This is not a high quality image.

Forget the wooden table! Where's the skin-tight t-shirt and provocative pose?

2013-04-23 Reply Admin

She's a wannabe politician in making. This is posturing at best, canvassing at worst. R.I.P Hugo Chavez!

2013-04-23 Reply Admin

Will:
[...] are usually the people at the coalface making the change.

"Coalface"? Please enlighten me.

chubertdev · 2013-04-23 Reply Admin

JJ:
Will:
[...] are usually the people at the coalface making the change.
"Coalface"? Please enlighten me.

I'm guessing that when you're at the furnace shoveling coal, you get coalface.

2013-04-24 Reply Admin

ih8u:
QJo:
TRWTF:
A horrible dread crept over John’s skin. “I must have left in some flags, or some settings- it’s pulling data from PerfTest..."

The exact same build that is deployed in Test should be able to be used without changes in Production. If you need to make amendments to the code base before you perform the Production deployment you're doing it wrong.

Exactly:

#ifdef TEST do_stuff(); #else do_somewhat_or_completely_different_stuff(); #endif

There's the causa (captcha) of your problem.

You captcha'd the causa susincto!

Seahen · 2013-04-24 Reply Admin

Aww, from the title I was hoping the story would be about a program barely alive. We can rebuild it. We have the technology. We can make it better than it was. Better. Stronger. Faster.

2013-04-24 Reply Admin

chubertdev:
JJ:
Will:
[...] are usually the people at the coalface making the change.
"Coalface"? Please enlighten me.

I'm guessing that when you're at the furnace shoveling coal, you get coalface.

It's from mining. The coalface is the edge of the coal seam -- the part actively being mined. So to be at the coalface would mean you're working right at the deepest part of the mine doing the real work...kinda similar to saying "on the front lines".

2013-04-29 Reply Admin

jay:
And by the way:
If you follow the rules, and things go badly, it's not your fault, it's the fault of the person who made up the rules.

If you break the rules, and things go badly, then it's your fault.

If you break the rules, and things go well, you may still get in trouble for breaking the rules.

So from the employee's point of view, the safe thing to do is always to follow the rules.
                | follow rules           | break rules |
----------------+------------------------+-------------+
turns out well  | you win             +5 | toss-up  -1 |
turns out badly | blame somebody else +1 | you lose -5 |
----------------+------------------------+-------------+
Clearly the best strategy from a game theory point of view is to follow the rules.

Only in a closed system.

2013-04-29 Reply Admin

Tom:
All that, and no one mentions the dumb-as-rocks sysadmin who says "No need to roll back, we'll just tweak the settings?"
Seriously? If one of my applications went tango-uniform, and the sysadmin said that to me, I'd knock him in to next week and kick his ass on Tuesday. There's a reason my deployment instructions include rollback instructions: so that if the deployment is @&#^ed for a cocked hat, you can ROLL IT BACK.

That's TRWTF, right there. CM mistakes happen, young turk developers screw up, managers push too hard to make an unrealistic schedule ("We have to do it quick, we don't have time to do it right.") But when you're in a bad state, you back out of it. You can't "tweak" your way out of a train wreck.

"You can't tweak your way out of a train wreck?"

Hot damn. That's brillant! It should be on a T-Shirt somewhere!

(I miss Irish Girl.)

2013-04-30 Reply Admin

Some Damn Yank:
notchulance:
This is why we need to bring back critical thinking in school. We're becoming a civilization of mindless drones who can't think outside the box.
^ This.
I find it hilarious that the quote and your reply reside inside a box.

[image]

2013-05-02 Reply Admin

I'm shocked that a WTF should turn out be a fuck up of some kind. How insightful of you.

Captcha is "nobis"...

2013-05-03 Reply Admin

SWIFT Interview questions on

http://testwithus.blogspot.in/p/swift.htm

For selenium solution visit http://testwithus.blogspot.in/p/blog-page.html

QTP Interview Questions. http://testwithus.blogspot.in/p/qtp-questions.html

www.searchyourpolicy.com

2013-05-09 Reply Admin

Oooh, politics on a tech story forum.

"Smoke 'em up, Johnny! Cancer just tickles a little. And don't forget to beat the crap out of anyone different from you on the school bus!"

2013-05-23 Reply Admin

ObiWayneKenobi:
I am continually amazed at how mindless people become when it comes to these things. There's a critical bug that needs to be fixed right now! But we have 3 layers of approval it has to go through before it can go, and one manager is on vacation, and nobody here is intelligent enough to do some critical thinking to fix it, so the bug stays because everyone is too afraid to deviate from the process.
This is why we need to bring back critical thinking in school. We're becoming a civilization of mindless drones who can't think outside the box.

That's not really critical thinking, it's a matter of "I don't want to get blamed" x 1000. They know that they take no blame from doing nothing and they might take blame from doing something. So they do nothing.

It's still pathetic, though.

The Six Million Dollar Patch

Leave a comment on “The Six Million Dollar Patch”