Comment On Testing Done Right

It’s been a rough couple weeks. Not only did I have all sorts of catching-up to do after Code PaLOUsa, but it also happened to be release week. And oh, do I hate release week. [expand full text]
« PrevPage 1 | Page 2 | Page 3 | Page 4 | Page 5Next »

Re: Testing Done Right

2011-03-23 12:18 • by Rob (unregistered)
341943 in reply to 341816
C-Octothorpe:
boog:
Anonymous:
I agree, I'd love to work with technically-minded testers, but I am begrudgingly speaking from experience. We only have one capable tester, the rest are rejects who only test because they don't have the skill to do anything else in IT.
This is what I've experienced too; it's easy to tell which testers actually know what they're doing. Unfortunately, management wrongly sees testing as a mindless (and profitless) task, so they usually try to hire testers from the local zoo.

Anonymous:
The monkey analogy is all too accurate, to be honest.
My advice: wear a raincoat.


... and goggles. Some of them have sniper aim...

SIDENOTE: are there really people who WANT to test? I mean, speaking as a developer looking at the role of QA, it just doesn't seem all that appealing to me. I am not the smartest guy by far, but from my personal experience, QA tends to attract people with the intelligence levels reserved for the mildly retarded (I'd say in the neighbourhood of 100, give or take 10 pts). This usually results in people three kinds of QA monkeys:

1) The new grad/career switch above-average smart person trying to get into software development. Some people take this path, which is fine, and in some corps a necessary step...

2) The shitty, change resistant x-developer who couldn't hash it anymore and (in)voluntarily goes into QA. This is more rare as pride usually takes over and they just quit and f*ck up another dev shop with their presence.

3) The mildly retarded sociopaths who don't have the intelligence or social skills necessary to make it in any other role IT related. They may be able to make something of themselves in another profession, but because they got mad skillz at Halo, they MUST be good at everything IT related, which keeps them in the QA gig.

I realize this sounds very elitist, which isn't my intention but this is just what I have noticed/pondered...


Maybe I'm some freak of nature; but I tend to enjoy things that other developers roll their eyes at. I have no preference between working on bugs in existing code vs. writing some new feature. From what I've seen, most people hate fixing bugs (particularly when it's code they didn't write). Likewise, I enjoy testing.

The problem is, it feels like my career options are greater as a software developer than a software tester. If I go to Dice or CareerBuilder or whatever, there are a lot of dev jobs I fit the profile for, but fewer QA type jobs. And the pay of the QA type jobs is lower (most of the time). And, with every year of additional development work I gain, the harder it seems to switch. An entry level QA job doesn't pay enough. The well paying QA job requires X years of experience in QA, and as a developer, I don't have that.

Re: Testing Done Right

2011-03-23 12:20 • by Yankee Doodle (unregistered)
341944 in reply to 341942
callcopse:
TRWTF is clearly insecure septic tanks trying to advertise their purportedly outsized genitalia in programming blog comments. Who's that for the benefit of then?


TRWTF is cockney rhyming slang.

Re: Testing Done Right

2011-03-23 12:21 • by hoodaticus
341945 in reply to 341943
Rob:
The problem is, it feels like my career options are greater as a software developer than a software tester. If I go to Dice or CareerBuilder or whatever, there are a lot of dev jobs I fit the profile for, but fewer QA type jobs. And the pay of the QA type jobs is lower (most of the time). And, with every year of additional development work I gain, the harder it seems to switch. An entry level QA job doesn't pay enough. The well paying QA job requires X years of experience in QA, and as a developer, I don't have that.
Just be thankful you were born with the gift of programming. Not everyone can do it - no matter how smart they are otherwise.

Re: Testing Done Right

2011-03-23 13:09 • by C-Octothorpe (unregistered)
341948 in reply to 341943
Rob:

The problem is, it feels like my career options are greater as a software developer than a software tester. If I go to Dice or CareerBuilder or whatever, there are a lot of dev jobs I fit the profile for, but fewer QA type jobs. And the pay of the QA type jobs is lower (most of the time). And, with every year of additional development work I gain, the harder it seems to switch. An entry level QA job doesn't pay enough. The well paying QA job requires X years of experience in QA, and as a developer, I don't have that.


That's the point I'm trying to make is that IF you can make more money and have greater career opportunities, why would you settle for QA (lower pay usually, dead-end job unless you go into mgmt)? I just leave it at "to each his/her own".

I know of a few people who went from doing development fulltime into IT security (white-hat hacking, etc.), which I can see, but I have yet to see a "good" developer (or person who can easily do development) go into QA.

Re: Testing Done Right

2011-03-23 13:12 • by C-Octothorpe (unregistered)
341949 in reply to 341948
C-Octothorpe:
Rob:

The problem is, it feels like my career options are greater as a software developer than a software tester. If I go to Dice or CareerBuilder or whatever, there are a lot of dev jobs I fit the profile for, but fewer QA type jobs. And the pay of the QA type jobs is lower (most of the time). And, with every year of additional development work I gain, the harder it seems to switch. An entry level QA job doesn't pay enough. The well paying QA job requires X years of experience in QA, and as a developer, I don't have that.


That's the point I'm trying to make is that IF you can make more money and have greater career opportunities, why would you settle for QA (lower pay usually, dead-end job unless you go into mgmt)? I just leave it at "to each his/her own".

I know of a few people who went from doing development fulltime into IT security (white-hat hacking, etc.), which I can see, but I have yet to see a "good" developer (or person who can easily do development, or more than a temp gig because it's cool (I would test a Mercedes SLS for free damnit!)) go into QA.


... sorry, in response to what boog said a little while back.

Re: Testing Done Right

2011-03-23 13:38 • by Tzimisce
I disagree. Unit testing is different from integration testing. Unit tests should be testing classes in isolation, and should not involve external resources (databases, file system, web services, etc.). At least for TDD, a primary purpose of unit tests is to drive design.

Permitting customers or managers to dictate low quality levels is unprofessional, in my opinion. Uncle Bob Martin has many presentations on this subject. Is it ethical for an electrician to do a shoddy job in a new housing development because the developers asked him to? To quote Kent Beck, "Quality is terrible as a control variable."

Re: Testing Done Right

2011-03-23 14:03 • by boog
341952 in reply to 341951
Tzimisce:
I disagree. Unit testing is different from integration testing. Unit tests should be testing classes in isolation, and should not involve external resources (databases, file system, web services, etc.).

Given Alex's definition of Integration Testing, automated unit tests fit squarely under that category. You may certainly disagree with his definition (or more specifically the "Integration" label he attached to the definition), but the whole point of automated unit testing is to test newly-integrated changes to the code, so Alex's relating of the two is appropriate.

Re: Testing Done Right

2011-03-23 14:54 • by The Sound of 1 Foot Tapping (unregistered)
Lazy Bastard:
It’s been a rough couple weeks. Not only did I have all sorts of catching-up to do after Code PaLOUsa, but it also happened to be release week. And oh, do I hate release week.

So, what's your freakin' exuse now?

Re: Testing Done Right

2011-03-23 15:07 • by Design Pattern (unregistered)
341954 in reply to 341953
The Sound of 1 Foot Tapping:

So, what's your freakin' ex use now?

So today Axel did not pass, but broke up with his drug-addicted girl friend?

CAPTCHA: mara - is this the name of the girl friend?

Re: Testing Done Right

2011-03-23 15:08 • by Nagesh
341955 in reply to 341954
Design Pattern:
The Sound of 1 Foot Tapping:

So, what's your freakin' ex use now?

So today Axel did not pass, but broke up with his drug-addicted girl friend?

CAPTCHA: mara - is this the name of the girl friend?


You come here like crack addict and make excuse for others?

Re: Testing Done Right

2011-03-23 15:25 • by HonoredMule
I'd just like to point out: it is still theoretically possible to get hit by a bus even if you never leave your house.

99.999%...

Re: Testing Done Right

2011-03-23 16:26 • by Nagesh
341957 in reply to 341956
HonoredMule:
I'd just like to point out: it is still theoretically possible to get hit by a bus even if you never leave your house.

99.999%...


<insert toy bus photo here>

Re: Testing Done Right

2011-03-23 16:28 • by helix (unregistered)
341958 in reply to 341873
i would use it for space to store trays - you know the big ones that go on your lap to eat dinner when you are in front of the telly alone

Re: Testing Done Right

2011-03-23 16:28 • by The Coward (unregistered)
341959 in reply to 341956
HonoredMule:
I'd just like to point out: it is still theoretically possible to get hit by a bus even if you never leave your house.

99.999%...


Or is it you hitting the bus? :)

Anyhow my experience with TDD is that people spend ages faffing about with writing tests and not actually getting anywhere fast. I knock together something and get it working fast, then consider writing tests which aid in "and how could this be done better?".

After all in just about every other creative human endeavour you start out with rough sketches then see how this suits your needs (architect designing a new house, artist making a new character, etc). Then you increase the resolution and accuracy as you progress along (almost like zooming in on a fractal). You don't get the architect and some builders in the middle of a field and start building. In addition you are adding "mass" to the system which makes it more time consuming to go back and change fundamentals (re-factoring) when you could be more lean. (Funny how Agile types get rid of most/all of the spec/analysis docs then end up writing them in a "cool" new way).

In the future I'd like to see something like JTAG where you do not write separate code for tests but have probes/tripwires in the real code (even part of the language? ADA95?) and then have datasets which can setup that and check values that pass by; rather than having test code which can be obscure in itself.

My favourite issue with tests is that they do not really say if your product actually works, they just say for this very specific dataset this is what will happen in this environment. I have seen many a testing suite that results in 100% green lights but actually fails in real use, simply because the test data is not aligned with the real world data/usage! (very apparent for data driven apps).

Lastly people who are 100% code coverage fundamentalists seem to build code that has little fault tolerance in it [in my experience anyways]. I always try to make a program/function fail gracefully and clean up after itself. It's all very well that exception been thrown because it is the "correct" thing to do. Doesn't help your users at 2am in the morning if it all just disappears into a black hole!

Test in reality are like the Chinese proverb about a man and the time:

A man with one watch knows the time.
A man with two watches is never quite sure.

How many people here change their tests/data to get things to pass? This is where a proper analysis/spec document explaining the why is required.

Ah well, let's wait and see what the next software development band wagon is!

Re: Testing Done Right

2011-03-23 17:09 • by DevsDontGetItEither (unregistered)

So you don't like testing at least some of your own code ... hm.
And don't like fixing bugs ... hm.
Both of which are needed in any -worthwhile- s/w project (one that is sure to be a steaming pile of dung without either of the above).

Explain to me, why you are a developer, then?

Re: Testing Done Right

2011-03-23 17:23 • by Juqiel (unregistered)
341961 in reply to 341929
Lone Star Male:
Anonymous:
Lone Star Male:
EPA:
TRWTF are American Texan refrigerators, obviously. Everything's always bigger in the US Texas...

FTFY...and when they say everything, they mean everything, ladies.

Yep, a perfect description of the Texan ego.

Well, it is hard to hide...if you know what I mean.

A Texan ego is hard to hide?

Re: Testing Done Right

2011-03-23 17:25 • by DeLos
I thought the defect was going to be that he bought a refrigerator too short. That gap at the top is unsightly.

Re: Testing Done Right

2011-03-23 17:27 • by Dehli Belly (unregistered)
341963 in reply to 341935
boog:
Dehli Belly:
I would have thought testing requires all sorts of testing by all sorts of different parties to give the product the best chance at quality. Technical only testing doesn't work, because technical people aren't necessarily looking for results to mean the same thing. "Broken" doesn't mean the same thing either, nor any one of a number of descriptions for problems that occur. This site brings many opf them to light....

That depends on the product and its potential users, but I'm not talking about "types" of testing anyway. Yeah, technical-only testing probably isn't enough; I don't think anyone said otherwise.

I'm talking specifically about the testers' ability to clearly report failing tests and defects. How do you expect to efficiently deal with bugs if you don't have any details about them?


The way it was expressed (from memory) suggested (to me at least) that having techos test would fix the problem because they could better articulate the problem.

<copout>
That said, I wasn't necessarily disagreeing or arguing either, merely discussing
</copout>

Re: Testing Done Right

2011-03-23 17:28 • by id10t (unregistered)
341964 in reply to 341939
anon:
Yes. Knuth would be an obvious example, as in at least one of his correspondance he notes "Beware of bugs in the above code; I have only proved it correct, not tried it."

Also, the amount of people who turn "i=i+1;" statements into "i++;" statements; run automagic code formatters w/o testing; and just plain old 'it runs once, so every line in this file is automagically good' kind of people


Didn't Knuth the Polar Bear die recently?

Re: Testing Done Right

2011-03-23 17:32 • by The Coward (unregistered)
341966 in reply to 341960
DevsDontGetItEither:

So you don't like testing at least some of your own code ... hm.
And don't like fixing bugs ... hm.
Both of which are needed in any -worthwhile- s/w project (one that is sure to be a steaming pile of dung without either of the above).

Explain to me, why you are a developer, then?


Not sure if you are addressing that to me or not, but anyhow I am not saying testing is bad, just that a) understand the limits of it and b) don't sit there spending a day writing tests for code that does not exist and has no form yet.

Us it as an aid to make the code better designed, sure, use it as a starting point, well not so much.

I think the issue is that requirements are soft at first but writing tests first makes it concrete and absolute without the ability to play with the form first. For example:

"I want a car and door handles about waist height"

Code world:
Test:
Door handle at x,y,z coordinate.
Code:
Build code with door handle at x,y,z coordinate.

Show to user(s),
"Yeah it is ok but a bit higher would be nice and more laid in"

Result:
Lot of "production" level code and tests to change.

Real world:
Make foam door quickly (hack and slash), stick block of foam as handle on it.

Show to user(s),
Let them move it about.

Result:
Cheap none production item which then can be realised later to production quality with hard set tests.

This is how most real things are made, inexpensive as possible first then once realised the form make to production quality with all quality tools to bear.


Interesting side point; Software via unit tests and IOC etc result in many more pieces of software with more interfaces and interactions. Real world production of items is always driving down the number of parts and interfaces as these are what make it more expensive/less reliable.

Perhaps software should be built twice, once shitty and quick, twice to production quality (and learning from mistakes). But hey who is going to pay for the second part? ;)

Re: Testing Done Right

2011-03-23 17:52 • by Matt Westwood
341967 in reply to 341958
helix:
i would use it for space to store trays - you know the big ones that go on your lap to eat dinner when you are in front of the telly alone


Nope, can't grasp the context. What is this "alone" you speak of?

Re: Testing Done Right

2011-03-23 18:05 • by boog
341968 in reply to 341963
Dehli Belly:
The way it was expressed (from memory) suggested (to me at least) that having techos test would fix the problem because they could better articulate the problem.

Not "techos" necessarily, but somewhat-intelligent life forms with even a mildly-technical mindset, much more so than trained monkeys. And not that they can articulate the problem itself (that is, not the cause of the bug), but rather
1) what the unexpected behavior was;
2) a screenshot or any error details presented, if any; and
3) the conditions that led to this behavior (actions, date/time, inputs, etc.).

The lesser-testers just send an email saying "it's not working", leaving you to figure out what that even means.

Dehli Belly:
<copout>
That said, I wasn't necessarily disagreeing or arguing either, merely discussing
</copout>
As was I, my communicative cohort. As was I.

Addendum (2011-03-23 18:12):
Also: Not that having smarter testers would "fix the problem", but rather that I prefer smarter testers over chimps.

Re: Testing Done Right

2011-03-23 18:08 • by Ho Hum (unregistered)
341969 in reply to 341943
Rob:
C-Octothorpe:
boog:
Anonymous:
I agree, I'd love to work with technically-minded testers, but I am begrudgingly speaking from experience. We only have one capable tester, the rest are rejects who only test because they don't have the skill to do anything else in IT.
This is what I've experienced too; it's easy to tell which testers actually know what they're doing. Unfortunately, management wrongly sees testing as a mindless (and profitless) task, so they usually try to hire testers from the local zoo.

Anonymous:
The monkey analogy is all too accurate, to be honest.
My advice: wear a raincoat.


... and goggles. Some of them have sniper aim...

SIDENOTE: are there really people who WANT to test? I mean, speaking as a developer looking at the role of QA, it just doesn't seem all that appealing to me. I am not the smartest guy by far, but from my personal experience, QA tends to attract people with the intelligence levels reserved for the mildly retarded (I'd say in the neighbourhood of 100, give or take 10 pts). This usually results in people three kinds of QA monkeys:

1) The new grad/career switch above-average smart person trying to get into software development. Some people take this path, which is fine, and in some corps a necessary step...

2) The shitty, change resistant x-developer who couldn't hash it anymore and (in)voluntarily goes into QA. This is more rare as pride usually takes over and they just quit and f*ck up another dev shop with their presence.

3) The mildly retarded sociopaths who don't have the intelligence or social skills necessary to make it in any other role IT related. They may be able to make something of themselves in another profession, but because they got mad skillz at Halo, they MUST be good at everything IT related, which keeps them in the QA gig.

I realize this sounds very elitist, which isn't my intention but this is just what I have noticed/pondered...


Maybe I'm some freak of nature; but I tend to enjoy things that other developers roll their eyes at. I have no preference between working on bugs in existing code vs. writing some new feature. From what I've seen, most people hate fixing bugs (particularly when it's code they didn't write). Likewise, I enjoy testing.

The problem is, it feels like my career options are greater as a software developer than a software tester. If I go to Dice or CareerBuilder or whatever, there are a lot of dev jobs I fit the profile for, but fewer QA type jobs. And the pay of the QA type jobs is lower (most of the time). And, with every year of additional development work I gain, the harder it seems to switch. An entry level QA job doesn't pay enough. The well paying QA job requires X years of experience in QA, and as a developer, I don't have that.


I tend to agree that I am equally happy in support as development (although development where I had full control to dictate design and technology choice might be interesting). I had a manger once who insisted that all developers prefer to be in development roles (I was arguing that I was more than happy not to be rotated between support and development). He found that bizarre because he claimed that people like development because they get moire exposure ("Wow, Look...we actually released something"). Frankly, I suspect support staff get far more credit because they fix problems that already exist...That is, they stop problems that are already affecting users (<cynicism>not introducing new ones</cynicism>). Clients don't appreciate a good product until it has proven itself a good product. By this time, development teams are long forgotten....

Re: Testing Done Right

2011-03-23 19:30 • by fatnino (unregistered)
341970 in reply to 341956

Re: Testing Done Right

2011-03-23 20:59 • by Gary S (unregistered)
341971 in reply to 341905
MadJo (professional software tester):


And you NEVER let a developer test his own stuff, because it'll be hard get a grip on the quality in that case.
No one is truly really critical on his or her own creations. "Oh, I know what I mean there. I'll just leave it in." (also testers shouldn't review their own testcases, let someone else do that)


What on earth are you talking about? Of course the developer has to test their own stuff. How else will they know they've actually done the job? Someone else should also test their stuff, certainly, but the developer has to be the first person to give it a thorough test.

One of the biggest scourges in this industry are developers who never test their work. "Oh, the unit tests / QA team will find any problems, so I don't need to bother". Too many times I've seen some idiot implement a repeater with paging controls and tell me it's done, and found that had they tried even ONCE to see if the paging worked, they would have realised that it didn't.

Developers need to be responsible for their work. That means they must test what they do thoroughly. By all means have additional testing to verify their work, but this business about never "letting" developers test is a complete load of horseshit. By the time the work gets to the testing team, bugs should be extremely hard to find or non-existent. That does not happen by accident.

Re: Testing Done Right

2011-03-24 01:09 • by Jason (unregistered)
I know many others have said similar things, but this seems to be becoming the bi-weekly WTF (maybe tri-weekly)....

Re: Testing Done Right

2011-03-24 02:06 • by Mathy (unregistered)
Great post, while I agree on the point you're trying to make that at best (with unlimited resources) you can be 99.99 to some finite degree confident and never 100%, the math nerd in me wants to point out that:

99.9999... = 100

100/9 = 11.111...
9 * (100/9) = 9 * (11.111...)
900/9 = 99.999...
100 = 99.999...
QED

Re: Testing Done Right

2011-03-24 06:26 • by MacFrog (unregistered)
Or, rather recently, from around where I live:



BTW: This is not funny. Two people died in this accident.

Re: Testing Done Right

2011-03-24 07:08 • by trwtf (unregistered)
341978 in reply to 341977
MacFrog:
Or, rather recently, from around where I live:



BTW: This is not funny. Two people died in this accident.

BWAHAHAHAHAHA, casualties! Wait, what?

Re: Testing Done Right

2011-03-24 08:09 • by Design Pattern (unregistered)
341979 in reply to 341957
Nagesh:
HonoredMule:
I'd just like to point out: it is still theoretically possible to get hit by a bus even if you never leave your house.

99.999%...


<insert toy bus photo here>


Toy bus photo here!

Re: Testing Done Right

2011-03-24 08:11 • by martin (unregistered)
minimize the codebase and to keep things as simple as possible, thereby reducing the number of components and the overall complexity. But that’s a whole different soapbox.


That's an interesting soapbox. I'd love to hear some thoughts on that one. Especially on telling the customer not to have another useless expensive feature.

Re: Testing Done Right

2011-03-24 08:34 • by minime (unregistered)
341981 in reply to 341816
C-Octothorpe:
SIDENOTE: are there really people who WANT to test?


I met a few, and they don't really want to test everything, but they have a certain interest in proving that what other people did was wrong, and they came up with really intresting ways to rape our systems.


Another thing that has not really being discussed here is that testing is usually one of the first things (after documentation) that goes over board when hard deadlines have to be met. And sometimes the deadlines can be so hard that you throw everything over board and have the go-live as an integration test... that dead people is the unfortunate thing called "life outside the ivory tower".

Re: Testing Done Right

2011-03-24 08:42 • by boog (unregistered)
341982 in reply to 341953
The Sound of 1 Foot Tapping:
Lazy Bastard:
It’s been a rough couple weeks. Not only did I have all sorts of catching-up to do after Code PaLOUsa, but it also happened to be release week. And oh, do I hate release week.

So, what's your freakin' exuse now?

I'm pretty sure I would strangle you before you got any further. Look, this is Alex's site, and if he wants to flush it down the toilet by never posting any articles, that's his perogative. If you don't like it, you can go start your own humour/software site.

BTW, if you do this, please post a link here. With the lack of articles lately, there's been nothing to do but turn to trolling.

Re: Testing Done Right

2011-03-24 08:50 • by The Explainer (unregistered)
341983 in reply to 341982
boog:
The Sound of 1 Foot Tapping:
Lazy Bastard:
It’s been a rough couple weeks. Not only did I have all sorts of catching-up to do after Code PaLOUsa, but it also happened to be release week. And oh, do I hate release week.

So, what's your freakin' exuse now?

I'm pretty sure I would strangle you before you got any further. Look, this is Alex's site, and if he wants to flush it down the toilet by never posting any articles, that's his perogative. If you don't like it, you can go start your own humour/software site.

BTW, if you do this, please post a link here. With the lack of articles lately, there's been nothing to do but turn to trolling.


Pretty much, lately it's been The Daily "WTF? No article again?"

Re: Testing Done Right

2011-03-24 08:51 • by Michael Jordan (unregistered)
341984 in reply to 341983
The Explainer:
boog:
The Sound of 1 Foot Tapping:
Lazy Bastard:
It’s been a rough couple weeks. Not only did I have all sorts of catching-up to do after Code PaLOUsa, but it also happened to be release week. And oh, do I hate release week.

So, what's your freakin' exuse now?

I'm pretty sure I would strangle you before you got any further. Look, this is Alex's site, and if he wants to flush it down the toilet by never posting any articles, that's his perogative. If you don't like it, you can go start your own humour/software site.

BTW, if you do this, please post a link here. With the lack of articles lately, there's been nothing to do but turn to trolling.


Pretty much, lately it's been The Daily "WTF? No article again?"


Oh, see, you thought "W" was for "What". It's actually for "When."

Re: Testing Done Right

2011-03-24 08:54 • by Name Your (unregistered)
341985 in reply to 341981
minime:
C-Octothorpe:
SIDENOTE: are there really people who WANT to test?


I met a few, and they don't really want to test everything, but they have a certain interest in proving that what other people did was wrong, and they came up with really intresting ways to rape our systems.


Another thing that has not really being discussed here is that testing is usually one of the first things (after documentation) that goes over board when hard deadlines have to be met. And sometimes the deadlines can be so hard that you throw everything over board and have the go-live as an integration test... that dead people is the unfortunate thing called "life outside the ivory tower".


I think you meant "dear people", unless you were talking about Therac-25 or something.

Re: Testing Done Right

2011-03-24 09:12 • by Roger Wolff (unregistered)
Alex, nice article...

But you're making it sound as if testing is only done to catch errors introduced by changes. As if the testers are supposed to ignore errors that slipped by during testing of the previous release. This of course is not intended or wanted.

Testing should ensure the quality of the upcoming release. Not just the changes from the last version to the upcoming version.

Suppose a tester finds a possible bug that has been in the codebase for ages. "won't fix" because it wasn't introduced with a change for the upcoming release? Absurd.

Re: Testing Done Right

2011-03-24 10:02 • by Pedantic Fool (unregistered)
342003 in reply to 341975
Jason:
I know many others have said similar things, but this seems to be becoming the bi-weekly WTF (maybe tri-weekly)....


http://grammar.quickanddirtytips.com/biweekly-versus-semiweekly.aspx

Although, you get 10 pedant points for using the four dotted ellipsis at the end of a sentence....

Re: Testing Done Right

2011-03-24 10:07 • by frits
I almost forgot about one of the most effective strategies for identifying defects. Have developers test each other's code. Make sure to match up developers that really dislike each other.

Re: Testing Done Right

2011-03-24 10:56 • by trtrwtf (unregistered)
342021 in reply to 341958
helix:
i would use it for space to store trays - you know the big ones that go on your lap to eat dinner when you are in front of the telly alone


It's still early, but that's the saddest thing I've read all day.
Don't you have a newspaper and a kitchen table?

Re: Testing Done Right

2011-03-24 11:29 • by Anonymous (unregistered)
342025 in reply to 342004
frits:
I almost forgot about one of the most effective strategies for identifying defects. Have developers test each other's code. Make sure to match up developers that really dislike each other.

I find that most offices have at least one person who is a total perfectionist and will bitch and moan at the slightest defect or any infraction of coding standards, however trivial. That person is generally pretty unpopular but they police the code better than any project manager could. I'm a little bit ashamed to admit that in this office, it's me.

Re: Testing Done Right

2011-03-24 11:41 • by Hatshepsut
342027 in reply to 342021
trtrwtf:
helix:
i would use it for space to store trays - you know the big ones that go on your lap to eat dinner when you are in front of the telly alone


It's still early, but that's the saddest thing I've read all day.
Don't you have a newspaper and a kitchen table?


The newspaper's a good idea to spread on the sofa to catch dribbles, but I'm not sure the kitchen table would fit on my lap.

Re: Testing Done Right

2011-03-24 16:21 • by Moug (unregistered)
Ok the real WTF in that is the picture of the Clock..
Wow thats ugly.

Re: Testing Done Right

2011-03-24 18:00 • by lf8 (unregistered)
342076 in reply to 342003
Pedantic Fool:
Jason:
I know many others have said similar things, but this seems to be becoming the bi-weekly WTF (maybe tri-weekly)....


http://grammar.quickanddirtytips.com/biweekly-versus-semiweekly.aspx

Although, you get 10 pedant points for using the four dotted ellipsis at the end of a sentence....


So, he's implying that the article is posted every two weeks not twice a week. Who cares? I think the point was that it seems a long time between drinks.

Shouldn't there be 3 dots not 4? Or was that your point

Re: Testing Done Right

2011-03-24 18:07 • by Jelly (unregistered)
342077 in reply to 342025
Anonymous:
frits:
I almost forgot about one of the most effective strategies for identifying defects. Have developers test each other's code. Make sure to match up developers that really dislike each other.

I find that most offices have at least one person who is a total perfectionist and will bitch and moan at the slightest defect or any infraction of coding standards, however trivial. That person is generally pretty unpopular but they police the code better than any project manager could. I'm a little bit ashamed to admit that in this office, it's me.


No offence intended, but in my experience the people who seem to be the biggest perfectionists about others' code are the ones that struggle to produce their own (I'm not saying this is necessarily the category you fit into, btw).

I have worked with many people who in code reviews would get pedantic about a miscellany of issues including misspellings in change descriptions, copyright needs updating, indenting slightly askew, insisting people identify where memory allocated was freed (which always made me think they couldn't find the spot themselves), insisting that pointers to freed memory be nulled, variable names etc.
Although some of these things may be good practice, it seemed that these people would like to appear productive at code-reviews to make up for the fact that there was rarely any of their code being reviewed (and when there was, it would almost always either be copied from elsewhere, or actually done by someone else (or at least, any issues that came about in reviews of their code were dismissed as somehow being someone else's fault)).

Re: Testing Done Right

2011-03-24 23:24 • by Pedantic Fool (unregistered)
342083 in reply to 342076
lf8:
Pedantic Fool:
Jason:
I know many others have said similar things, but this seems to be becoming the bi-weekly WTF (maybe tri-weekly)....


http://grammar.quickanddirtytips.com/biweekly-versus-semiweekly.aspx

Although, you get 10 pedant points for using the four dotted ellipsis at the end of a sentence....


So, he's implying that the article is posted every two weeks not twice a week. Who cares? I think the point was that it seems a long time between drinks.

Shouldn't there be 3 dots not 4? Or was that your point

Didn't you read my name? BTW-It's definitely 4 dots at the end of a sentence.

Re: Testing Done Right

2011-03-25 08:02 • by Anonymous (unregistered)
342093 in reply to 342077
Jelly:
Anonymous:
frits:
I almost forgot about one of the most effective strategies for identifying defects. Have developers test each other's code. Make sure to match up developers that really dislike each other.

I find that most offices have at least one person who is a total perfectionist and will bitch and moan at the slightest defect or any infraction of coding standards, however trivial. That person is generally pretty unpopular but they police the code better than any project manager could. I'm a little bit ashamed to admit that in this office, it's me.


No offence intended, but in my experience the people who seem to be the biggest perfectionists about others' code are the ones that struggle to produce their own (I'm not saying this is necessarily the category you fit into, btw).

I have worked with many people who in code reviews would get pedantic about a miscellany of issues including misspellings in change descriptions, copyright needs updating, indenting slightly askew, insisting people identify where memory allocated was freed (which always made me think they couldn't find the spot themselves), insisting that pointers to freed memory be nulled, variable names etc.
Although some of these things may be good practice, it seemed that these people would like to appear productive at code-reviews to make up for the fact that there was rarely any of their code being reviewed (and when there was, it would almost always either be copied from elsewhere, or actually done by someone else (or at least, any issues that came about in reviews of their code were dismissed as somehow being someone else's fault)).

I hear you but there is a difference between pedantic and assiduous. Policing non-functional stuff (indenting, variable names etc) does not improve the quality of the code in any way, it's just not helpful. Similarly, one person's coding style is no better than another's and it's not helpful to push "your" style onto someone else - as long as everyone follows the coding standards it doesn't matter. As far as I'm concerned, if it doesn't affect the code in terms of function or readability/maintainability, then it doesn't matter. I certainly don't think I'm one of those pedantic overseers but you'd have to ask my colleagues!

Re: Testing Done Right

2011-03-25 10:59 • by Tim Barrass (unregistered)
The most robust system I every helped design and build (handling continuous large scale transfers of data between academic institutions) was robust because it assumed from the outset that every single operation would fail.

Actually, it wasn't so much an assumption as a statement of fact about the components we were building our system from.

This meant our state model for the system's workflow had to be as tight as could be, and that we explicitly looked to handle failures.

Re: Testing Done Right

2011-03-25 15:35 • by The Coward (unregistered)
342172 in reply to 342122
One thing that annoys me are the 100% code coverage zealots. Ok, test "critical" bits, but checking single line returns that return a member [variable] is a bit pointless. More so with Mock as you are returning the fake item which you know is valid or invalid.

Usually they are very easy to break in the real world, pass in a negative number/nan/null and watch the house of cards come crashing down.

I'd rather have more intelligent targeted tests instead of paint by numbers.

Hell in industry they don't test every single thing, they sample batches (e.g. bakers dozen). Unless of course you are making bolts for a nuclear reactor then you x-ray each of those bad boys!

I prefer to spend more time on battle hardening code for the real world which involves having real world data going into the system via the normal data channels, not through a mock or stand alone bit. (I suppose more like integration test but not quite).

Bottom line, qualitative vs quantitative testing.

Re: Testing Done Right

2011-03-26 21:05 • by hlovdal (unregistered)
The only way to completely avoid the inherent risk of change is to avoid change altogether


That is not true. Or rather, the sentence in isolation is true since it only covers "the inherent risk of change", but the overall message is not. The text have two assumptions*:

1) Changing code implies risk - True
2) Not changing code is risk free - False!

and draws as a conclusion that "change is bad" from a risk perspective.

This is wrong. For instance changing the break pads on your car is not risk free. You might have bought wrong replacements or you might put them on incorrectly. However not changing the break pads is not risk free either (especially if you can hear them scream when being used!).

If you discover a off by one error in the source code, not correcting that bug might very well be more risky than correcting it. There is no way to say in a objective way; specific judgment is always required.

I am in no way saying that change is risk free. And I fully buy that even the most innocent change might turn out to have some unintended consequence. In fact even just changing comments might have a impact (not very likely but if the comment is shorter/longer the following __LINE__ tokens will change values and now might be one digit more or less. If such a token is made into a string by the pre-processor the string is now one byte smaller/larger which might trigger that some nearby code is moved into a different memory segment. And that might have a non-negligible run-time impact).

But to assume that not changing the code is risk free is just so utterly wrong.

You should always aim for minimizing risk during maintenance, and that is NOT the same as minimizing change!

----------------------------

*
My interpretation of it, great if I am wrong on this but then the text is very imprecise in my opinion.
« PrevPage 1 | Page 2 | Page 3 | Page 4 | Page 5Next »

Add Comment