The Daily WTF: Curious Perversions in Information Technology

Power Troll · 2011-03-22 Reply Admin

Oh my. frits?

Anyway, while I agree that 100% code coverage is meaningless when test defects exist, is it simply a gestalt "feeling" about when your code is good to go, or what?

2011-03-22 Reply Admin

Someone needs to explain that last bit to my boss. Badly.

2011-03-22 Reply Admin

Damn, that's a nice fridge. Anyway, I agree with the sentiment and I always unit test functionality that cannot be adequately exercised by an end user. But everything else I leave to the chimps in the testing department. Why make senior developers do the work of a trained monkey? There's a reason we get paid twice as much as them, you know.

2011-03-22 Reply Admin

So Plato spoke latin huh? It was Juvenal.

2011-03-22 Reply Admin

Just checking whether I also need to type in " " to get a linebreak in my comment.

2011-03-22 Reply Admin

"At best (i.e., with unlimited resources), you can be 99.999…% confident that there will be no defects in production."

So, 100% confident then.

2011-03-22 Reply Admin

One real WTF is the number of developers (yes, 341777, I'm looking at you) who still think unit testing is a testing technique, rather than a programming one.

boog · 2011-03-22 Reply Admin

Change Impact – the estimated scope of a given change; this varies from change to change and, like testing, is always a “good enough” estimate, as even a seemingly simple change could bring down the entire system

I think about this every time somebody tells me to "just refactor".

2011-03-22 Reply Admin

SpasticWeasel:
So Plato spoke latin huh? It was Juvenal.

No, you're juvenile!

2011-03-22 Reply Admin

Even if you have coverage of 100% of the lines of code that doesn't mean you have covered 100% of the code paths. In fact, since the number of code paths through the entire code base grows exponentially you have covered some vanishingly small percentage of the code paths. For example, it is pretty easy to get 100% coverage of the following lines without uncovering the null pointer exception

if (x <= 0) { obj = NULL; } if (x >= 0) { obj.doSomething(); }

2011-03-22 Reply Admin

I have a problem with this:

Integration Testing – does the program function?

What does "the program function" means ? Doesn't crash ? That is easy. But what if just behave differently from expected ? Than, what is expected ? What is expected is define in the functional specifications, so what is the real difference between integration and functional testing ?

My personal definition, that i am using in my work in a big IT departement, is that integration test verify that a codebase correspond to a technical design, i.e. that the different modules interacts as the architect/developer decided, while the functional tests verify that the design and the code actually correspond to the functional requirements.

Opinions ?

2011-03-22 Reply Admin

As a QA Monkey I should be offended, but then I realized that programmers use all their salary to buy dates while I use mine for bananas.

frits · 2011-03-22 Reply Admin

Power Troll:
Oh my. frits?
Anyway, while I agree that 100% code coverage is meaningless when test defects exist, is it simply a gestalt "feeling" about when your code is good to go, or what?

You rang?

boog · 2011-03-22 Reply Admin

dadasd:
One real WTF is the number of developers (yes, 341777, I'm looking at you) who still think unit testing is a testing technique, rather than a programming one.

Why can't it be both?

2011-03-22 Reply Admin

Ed:
Someone needs to explain that last bit to my boss. Badly.

Why explain it to him badly?

boog · 2011-03-22 Reply Admin

Anonymous:
Anyway, I agree with the sentiment and I always unit test functionality that cannot be adequately exercised by an end user. But everything else I leave to the chimps in the testing department. Why make senior developers do the work of a trained monkey?

I've found that as a developer, I'd much prefer a testing department comprised of smart humans with strong technical skills, rather than chimps. With technical testers, you get surprisingly-useful reports about defects and failing tests, making it much easier to identify the problem and correct it.

With trained monkeys, you only get phone calls saying "program no work; fix it". The flying poo can become a bit of a problem too.

Hatshepsut · 2011-03-22 Reply Admin

An interesting definition of Acceptance Testing ("formal or informal testing to ensure that functional requirements as implemented are valid and meet the business need").

Where I come from, Acceptance Testing verifies that the delivered product meets the contractually agreed requirements. Verification that these requirements meet the business need is a separate matter.

Also interesting is the designation of functional and non-functional requirements - a concept that seems to come from a strongly data-processing oriented background, rather than, say, a system-control oriented background, in which performance is often a critical functional requirement.

dohpaz42 · 2011-03-22 Reply Admin

Maurizio:
I have a problem with this: > Integration Testing – does the program function?
What does "the program function" means ? Doesn't crash ? That is easy. But what if just behave differently from expected ? Than, what is expected ? What is expected is define in the functional specifications, so what is the real difference between integration and functional testing ?

My personal definition, that i am using in my work in a big IT departement, is that integration test verify that a codebase correspond to a technical design, i.e. that the different modules interacts as the architect/developer decided, while the functional tests verify that the design and the code actually correspond to the functional requirements.

Opinions ?

Testing is such a subjective area of programming, that it means different things to different people/businesses/departments. The question, "Does the program function?" could mean "Does it crash?", or "Does it do XYZ, ZYX, or some other functionality?"

Before you test, you first have to define your tests. But, to define your tests, you have to define what is a reasonable outcome for the test you want to write. Is there more than one way to access a piece of code? Does it produce more than one type of output - or any output, for that matter?

Testing is almost like software design; you have to sit down and plan out what you want to test, and how you're going to implement it. Which, unfortunately, to most business types is a 100% (or 99.(9)%) waste of time and effort.

2011-03-22 Reply Admin

I'm throwing a WtfNotFunny exception.

neminem · 2011-03-22 Reply Admin

Tim:
In fact, since the number of code paths through the entire code base grows exponentially you have covered some vanishingly small percentage of the code paths.

I'd argue anytime your input comes from the user or from another system you have no control over the output of, you've covered exactly 0%. A finite fraction of an infinite is, after all, precisely nothing. (We once found a major blocking bug in a process iff the text input to the process started with a lowercase 'p'. That was a fun one.)

But I'm happy to work at a computer that distinguishes between devs and testers; we certainly are responsible for testing the effects of our own code before checking in changes (that would be your #1), but the "test team" consists of people that were hired for that purpose. I sort of thought it was like that everywhere (well, everywhere where the whole company isn't a couple devs.)

frits · 2011-03-22 Reply Admin

The 100% code coverage metric is so dumb. Have these people never heard of permutation? I assume that 100% of the code is tested at least informally. I mean do some developers really write code that they've never run before releasing it?

Hatshepsut · 2011-03-22 Reply Admin

Anonymous:
Damn, that's a nice fridge.

Shame about the furniture.

And the clouds on the ceiling.

2011-03-22 Reply Admin

I mean do some developers really write code that they've never run before releasing it?

Yes. Unfortunately, the amount of testing a developer does tends to have a negative correlation with the amount of testing their code actually needs. There's usually no clearer sign of a developer who belongs on the dole queue than one who thinks their code will work first time.

2011-03-22 Reply Admin

boog:
Anonymous:
Anyway, I agree with the sentiment and I always unit test functionality that cannot be adequately exercised by an end user. But everything else I leave to the chimps in the testing department. Why make senior developers do the work of a trained monkey?
I've found that as a developer, I'd much prefer a testing department comprised of smart humans with strong technical skills, rather than chimps. With technical testers, you get surprisingly-useful reports about defects and failing tests, making it much easier to identify the problem and correct it.
With trained monkeys, you only get phone calls saying "program no work; fix it". The flying poo can become a bit of a problem too.

I agree, I'd love to work with technically-minded testers, but I am begrudgingly speaking from experience. We only have one capable tester, the rest are rejects who only test because they don't have the skill to do anything else in IT. They sit at their desks grinding the organ according to the pre-defined scripts authored by the one guy that actually knows what he's doing. The monkey analogy is all too accurate, to be honest.

2011-03-22 Reply Admin

Jon:
I'm throwing a WtfNotFunny exception.

Implying that this is somehow an exception to the rule.

Bazzzzing!

2011-03-22 Reply Admin

Anonymous:
boog:
Anonymous:
Anyway, I agree with the sentiment and I always unit test functionality that cannot be adequately exercised by an end user. But everything else I leave to the chimps in the testing department. Why make senior developers do the work of a trained monkey?
I've found that as a developer, I'd much prefer a testing department comprised of smart humans with strong technical skills, rather than chimps. With technical testers, you get surprisingly-useful reports about defects and failing tests, making it much easier to identify the problem and correct it.
With trained monkeys, you only get phone calls saying "program no work; fix it". The flying poo can become a bit of a problem too.
I agree, I'd love to work with technically-minded testers, but I am begrudgingly speaking from experience. We only have one capable tester, the rest are rejects who only test because they don't have the skill to do anything else in IT. They sit at their desks grinding the organ according to the pre-defined scripts authored by the one guy that actually knows what he's doing. The monkey analogy is all too accurate, to be honest.

Pay peanuts, etc. Sack your team, and hire people who actually want to test, have technical skills and take it seriously as both a career and a technical field. Works for Microsoft.

boog · 2011-03-22 Reply Admin

Anonymous:
I agree, I'd love to work with technically-minded testers, but I am begrudgingly speaking from experience. We only have one capable tester, the rest are rejects who only test because they don't have the skill to do anything else in IT.

This is what I've experienced too; it's easy to tell which testers actually know what they're doing. Unfortunately, management wrongly sees testing as a mindless (and profitless) task, so they usually try to hire testers from the local zoo.

Anonymous:
The monkey analogy is all too accurate, to be honest.

My advice: wear a raincoat.

dohpaz42 · 2011-03-22 Reply Admin

frits:
The 100% code coverage metric is so dumb. Have these people never heard of permutation? I assume that 100% of the code is tested at least informally. I mean do some developers really write code that they've never run before releasing it?

I don't think that it's dumb. I think that it is misunderstood. 100% code coverage, to me, means that every class/function/method has a corresponding test. The real questions are: Are the tests sufficient? Are the tests meaningful?

Having 100% code coverage is a worthy goal to attempt to achieve, so long as you don't just try to write tests for the sake of writing tests. Can you possibly cover every single permutation of error that could possibly occur? No, and you shouldn't necessarily try (it goes back to what Alex said about the cost/effort going up as you get closer to 100%).

As an example, if I wrote a class that had three methods in it, then to me it is reasonable that to achieve 100% code coverage I would need three unit tests. If I add a new method, and I add a new unit test, then I'm still at 100% code coverage.

Do these tests test for every single possible outcome? Probably not. Should I care? Yes. Will I write a test for every single possible outcome? Hell no.

To me there are two types of development: test-driven development, where I write tests [theoretically] before I write code - this is a practice that helps to shape the program, while simultaneously giving me code coverage AND documentation; and, exception-driven development, where tests are written to reproduce bugs as they are found. This helps to document known issues, reproduce said issues, and provide future test vectors to insure that future changes don't re-introduce already fixed bugs.

2011-03-22 Reply Admin

A few years ago my old company off-shored all their testing to India. Not the development, just testing - so the off-shore team were writing and running scripts for a black-box that they didn't even begin to understand. Three bug-ridden releases later and... well, let's just say that company isn't around anymore.

Coyne · 2011-03-22 Reply Admin

I agree testing designed to reach some level of quality that will never be 100%.

Where I often see a lack in software is what I would call recoverability: The ability to correct a problem after it has occurred.

I'm generally a cautious individual. Experience has demonstrated that I should always have a fallback plan, which is really a chain of defenses thing.

If the application has no recoverability, then there is no fallback: The only thing between "everything is good" and "absolute disaster" is a fence called "everything works perfectly". When designing applications, one of the things that should always be done is to consider, "What if this doesn't work? What would be our fallback plan?"

Because, if you don't think about that and plan for that, then one day something doesn't work perfectly and you find yourself in absolute disaster land because you have no other line of defense. That is actually the source of some really good stories (in here as well as in other places). I'll relate one:

(At a previous life.) We bought a 3rd-party product for donation management. The builders of that product had a really interesting way of handling errors: They ignored all errors.
One of the processes was the daily account apply. You entered incoming donations in a batch; the apply process would then read the batch, update the accounts and delete the batch.

On the disaster day in question, the accounts table reached size limit part way (early on) through the processing of the batch and, since the developers ignored such mundane messages from the database as "the table is full and I can't insert this now", the process blythely continued on.

Then it deleted the batch.

No fallback. No way to recover the batch and so an entire day of entry by the user was lost.

Okay, so now let's create a fallback. That's hard, right? No, in this case actually it isn't: The solution is to back up the entire database before running the apply process. Every single time a batch is to be applied! That way, if something goes wrong, you fix the problem, restore, rerun and everything is cool.

...and usually, fallback is just like that. It mostly consists of one single element that I often see omitted: Keep the input state so that rerun is possible. There are "bazillions" of ways to do that; take your own pick.

But some people like to live on the edge and depend on the application doing everything right, and when it doesn't, well, glad I'm not them.

2011-03-22 Reply Admin

boog:
Unfortunately, management wrongly sees testing as a mindless (and profitless) task, so they usually try to hire testers from the local zoo.

Absolutely true, this is pretty much the crux of the problem I think.

boog:
My advice: wear a raincoat.

My screen is now intimately familiar with my last mouthful of coffee, thanks!

2011-03-22 Reply Admin

The Ancient Programmer:
Ed:
Someone needs to explain that last bit to my boss. Badly.

Why explain it to him badly?

If you explain it badly, perhaps he will have a gooder understanding.

2011-03-22 Reply Admin

Plato:
quis custodiet ipsos custodes?

... and all the classes and procedures will look up and shout "Test us!" ... and I'll look down and whisper "No."

2011-03-22 Reply Admin

boog:
Change Impact – the estimated scope of a given change; this varies from change to change and, like testing, is always a “good enough” estimate, as even a seemingly simple change could bring down the entire system
I think about this every time somebody tells me to "just refactor".

If a seemingly simple change can bring down the entire system, there's something fundamentally wrong with that system and it's ripe for a bit of guerrilla refactoring.

2011-03-22 Reply Admin

The Ancient Programmer:
Ed:
Someone needs to explain that last bit to my boss. Badly.

Why explain it to him badly?

Waste of time explaining it to him well. Pearls before swine.

2011-03-22 Reply Admin

I would like to sub scribe to this the ory about never leaving the house to avoid getting hit by a bus.

2011-03-22 Reply Admin

QJ:
boog:
Change Impact – the estimated scope of a given change; this varies from change to change and, like testing, is always a “good enough” estimate, as even a seemingly simple change could bring down the entire system
I think about this every time somebody tells me to "just refactor".

If a seemingly simple change can bring down the entire system, there's something fundamentally wrong with that system and it's ripe for a bit of gorilla refactoring.

FTFY

2011-03-22 Reply Admin

frits:
The 100% code coverage metric is so dumb. Have these people never heard of permutation? I assume that 100% of the code is tested at least informally. I mean do some developers really write code that they've never run before releasing it?

In some of the places I've worked I've had to opine that it would be nice if the developers checked whether their code actually compiles in the first place.

2011-03-22 Reply Admin

boog:
Anonymous:
I agree, I'd love to work with technically-minded testers, but I am begrudgingly speaking from experience. We only have one capable tester, the rest are rejects who only test because they don't have the skill to do anything else in IT.
This is what I've experienced too; it's easy to tell which testers actually know what they're doing. Unfortunately, management wrongly sees testing as a mindless (and profitless) task, so they usually try to hire testers from the local zoo.
Anonymous:
The monkey analogy is all too accurate, to be honest.
My advice: wear a raincoat.

... and goggles. Some of them have sniper aim...

SIDENOTE: are there really people who WANT to test? I mean, speaking as a developer looking at the role of QA, it just doesn't seem all that appealing to me. I am not the smartest guy by far, but from my personal experience, QA tends to attract people with the intelligence levels reserved for the mildly retarded (I'd say in the neighbourhood of 100, give or take 10 pts). This usually results in people three kinds of QA monkeys:

The new grad/career switch above-average smart person trying to get into software development. Some people take this path, which is fine, and in some corps a necessary step...
The shitty, change resistant x-developer who couldn't hash it anymore and (in)voluntarily goes into QA. This is more rare as pride usually takes over and they just quit and f*ck up another dev shop with their presence.
The mildly retarded sociopaths who don't have the intelligence or social skills necessary to make it in any other role IT related. They may be able to make something of themselves in another profession, but because they got mad skillz at Halo, they MUST be good at everything IT related, which keeps them in the QA gig.

I realize this sounds very elitist, which isn't my intention but this is just what I have noticed/pondered...

2011-03-22 Reply Admin

Brass Monkey:
QJ:
boog:
Change Impact – the estimated scope of a given change; this varies from change to change and, like testing, is always a “good enough” estimate, as even a seemingly simple change could bring down the entire system
I think about this every time somebody tells me to "just refactor".

If a seemingly simple change can bring down the entire system, there's something fundamentally wrong with that system and it's ripe for a bit of gorilla refactoring.
FTFY

+1! Sprayed coffee.

2011-03-22 Reply Admin

neminem:
Tim:
In fact, since the number of code paths through the entire code base grows exponentially you have covered some vanishingly small percentage of the code paths.
I'd argue anytime your input comes from the user or from another system you have no control over the output of, you've covered exactly 0%. A finite fraction of an infinite is, after all, precisely nothing. (We once found a major blocking bug in a process iff the text input to the process started with a lowercase 'p'. That was a fun one.)
But I'm happy to work at a computer that distinguishes between devs and testers; we certainly are responsible for testing the effects of our own code before checking in changes (that would be your #1), but the "test team" consists of people that were hired for that purpose. I sort of thought it was like that everywhere (well, everywhere where the whole company isn't a couple devs.)

There's a difference between "all possible code paths" and "all possible inputs".

Pedantic digression: The number of possible inputs is not infinite. There are always limits on the maximum size of an integer or lenght of a string, etc. So while the number of possible inputs to most programs is huge, it is not infinite.

Suppose I write (I'll use Java just because that's what I write the most, all due apologies to the VB and PHP folks):

String addperiod(String s)
{
  if (s.length()==0)
    return s;
  else if (s.endsWith("."))
    return s;
  else
    return s+".";
}

There are an extremely large number of possible inputs. But there are only three obvious code paths: empty string, string ending with period, and "other". There are at least two other not-quite-obvious code paths: s==null and s.length==maximum length for a string. Maybe there are some other special cases that would cause trouble. But for a test of this function to be thorough, we would not need to try "a", "b", "c", ... "z", "aa", "ab", ... etc for billions and billions of possible values.

That said, where we regularly get burned on testing is when we don't consider some case that turns out to create a not-so-obvious code path. Like, our code screws up when the customer is named "Null" or crazy things like that.

2011-03-22 Reply Admin

SIDENOTE: are there really people who WANT to test?

Testing, the way it's usually defined? Absolutely not. But with the right job description, testing can approach pure hacking: make this program break, by any means necessary. I've certainly had fun taking an hour or two away from my own code to find the security, performance, etc. holes in my colleagues', and I can see that being an enjoyable full-time job.

2011-03-22 Reply Admin

C-Octothorpe:
boog:
Anonymous:
I agree, I'd love to work with technically-minded testers, but I am begrudgingly speaking from experience. We only have one capable tester, the rest are rejects who only test because they don't have the skill to do anything else in IT.
This is what I've experienced too; it's easy to tell which testers actually know what they're doing. Unfortunately, management wrongly sees testing as a mindless (and profitless) task, so they usually try to hire testers from the local zoo.
Anonymous:
The monkey analogy is all too accurate, to be honest.
My advice: wear a raincoat.

... and goggles. Some of them have sniper aim...

SIDENOTE: are there really people who WANT to test? I mean, speaking as a developer looking at the role of QA, it just doesn't seem all that appealing to me. I am not the smartest guy by far, but from my personal experience, QA tends to attract people with the intelligence levels reserved for the mildly retarded (I'd say in the neighbourhood of 100, give or take 10 pts). This usually results in people three kinds of QA monkeys:

The new grad/career switch above-average smart person trying to get into software development. Some people take this path, which is fine, and in some corps a necessary step...

The shitty, change resistant x-developer who couldn't hash it anymore and (in)voluntarily goes into QA. This is more rare as pride usually takes over and they just quit and f*ck up another dev shop with their presence.

The mildly retarded sociopaths who don't have the intelligence or social skills necessary to make it in any other role IT related. They may be able to make something of themselves in another profession, but because they got mad skillz at Halo, they MUST be good at everything IT related, which keeps them in the QA gig.

I realize this sounds very elitist, which isn't my intention but this is just what I have noticed/pondered...

If testing is limited to "Click x. See if actual result matches expected result" then no, I doubt many people do. On the other hand, if it's "Build an automated framework for this system, then go to the architecture committee meeting to talk about testability. After that, meet with the product manager and BA to distill their requirements into Cucumber and then spend the afternoon on exploratory testing trying to break the new system" then why not? Done properly, QA is as much of an intellectual challenge and just as rewarding as development. The problem is that most testers are crap, and most test managers are worse, so they end up with the former definition of testing.

2011-03-22 Reply Admin

Hatshepsut:
An interesting definition of Acceptance Testing ("formal or informal testing to ensure that functional requirements as implemented are valid and meet the business need").
Where I come from, Acceptance Testing verifies that the delivered product meets the contractually agreed requirements. Verification that these requirements meet the business need is a separate matter.

Also interesting is the designation of functional and non-functional requirements - a concept that seems to come from a strongly data-processing oriented background, rather than, say, a system-control oriented background, in which performance is often a critical functional requirement.

I don't think you understand what functional and non-functional means in the context of software requirements. The fact that it is critical does not mean it becomes a functional requirement, it means it has to be met for the system to work correctly.

Also some companies manufacture and sell software themselves! i.e. you can't just say; "that's what it says in the requirements". Not all software development is outsourced. For example a computer game has to go through "Acceptance Testing" with the people who are likely to use it e.g. so called "beta testing".

2011-03-22 Reply Admin

Hatshepsut:
An interesting definition of Acceptance Testing ("formal or informal testing to ensure that functional requirements as implemented are valid and meet the business need").
Where I come from, Acceptance Testing verifies that the delivered product meets the contractually agreed requirements. Verification that these requirements meet the business need is a separate matter.

Also interesting is the designation of functional and non-functional requirements - a concept that seems to come from a strongly data-processing oriented background, rather than, say, a system-control oriented background, in which performance is often a critical functional requirement.

From someone who has worked in the defense contractor environment for a few years now, this is very much a truth. Whether or not the item/program meets the business need of the customer is not as important as whether that item/program meets the contract requirements. You could build the most sophisticated fighter jet around for $8 million a jet, but if it doesn't meet the contract requirements then the military could care less if it saves money/lives/time. The same is true for software, as long as it meets the contract requirements (even if it's just barely within those requirements) then the software is accepted, buggy interface and occasional crashes along with it.

Fortunately (at least in most cases) when it comes to the space program, just because it meets the contract requirements isn't enough, especially when it comes to man-rated spacecraft and space probes that'll be working for a decade or two (for example, the Cassini Saturn probe and the MESSENGER Mercury probe.) "Just good enough" doesn't cut it in situations like that, so the testing that is done is to ensure that things work, they work right and that they'll continue working in the future.

Then again, those probes undergo the most intense real-world testing/use that anyone could envision. But that's after delivery/launch. chuckle

2011-03-22 Reply Admin

C-Octothorpe:
boog:
My advice: wear a raincoat.

... and goggles. Some of them have sniper aim...

SIDENOTE: are there really people who WANT to test? I mean, speaking as a developer looking at the role of QA, it just doesn't seem all that appealing to me. I am not the smartest guy by far, but from my personal experience, QA tends to attract people with the intelligence levels reserved for the mildly retarded (I'd say in the neighbourhood of 100, give or take 10 pts). This usually results in people three kinds of QA monkeys:

The new grad/career switch above-average smart person trying to get into software development. Some people take this path, which is fine, and in some corps a necessary step...

The shitty, change resistant x-developer who couldn't hash it anymore and (in)voluntarily goes into QA. This is more rare as pride usually takes over and they just quit and f*ck up another dev shop with their presence.

The mildly retarded sociopaths who don't have the intelligence or social skills necessary to make it in any other role IT related. They may be able to make something of themselves in another profession, but because they got mad skillz at Halo, they MUST be good at everything IT related, which keeps them in the QA gig.

I realize this sounds very elitist, which isn't my intention but this is just what I have noticed/pondered...

If there is a trick to decent QA testing, its to Choose to do it in the first place. Choose to restrict code submits to only those being accompanied by functional unit tests. Choose to use a system of shared peer review for said code submits. Choose to have a continuous and scheduled build of your product, and to insist that fixing a broken build takes priority. Choose to have representatives from the coding teams participate in the final QA process. Choose to get signoff from all teams before releasing. Choose to have feature freezes, periodically-reviewed API documentation and believable deadlines. Choose a risk free release cycle. Choose QA for everyone (especially the managers).

Captcha was of no consequat.

2011-03-22 Reply Admin

JadedTester:
Pay peanuts, etc. Sack your team, and hire people who actually want to test, have technical skills and take it seriously as both a career and a technical field. Works for Microsoft.

And here I thought Microsoft's beta testers were the general public once the RTM was issued out. *gryn*

2011-03-22 Reply Admin

frits:
I mean do some developers really write code that they've never run before releasing it?

sadly, yes

dkf · 2011-03-22 Reply Admin

C-Octothorpe:
SIDENOTE: are there really people who WANT to test?

Yes, but it really does depend on whether you're starting out from the position of having tests already defined. When you're working with an already-highly-tested codebase that's got a good support framework in place, it's quite nice to focus strongly on TDD and trying to ensure that all new code paths that you add are fully tested. (Hint: it can result in the most amazing hacks to reliably trigger particularly awkward cases.)

But if the code has very few tests, or if you're doing integration tests across a whole mess of dependencies, testing is a real drag; huge amounts of work for little reward.

2011-03-22 Reply Admin

Alex Papadimoulis:
no matter how hard you try, a definitive answer is impossible. At best (i.e., with unlimited resources), you can be 99.999…% confident that there will be no defects in production

but but 99.999...% IS 100%

Testing Done Right

Leave a comment on “Testing Done Right”