- Feature Articles
- CodeSOD
- Error'd
- Forums
-
Other Articles
- Random Article
- Other Series
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
Even better, they already had a good understanding of how the software was supposed to work when they went about their testing responsibilities.
Admin
Testing doesn't help at all if the underlying quality of the code is just bad (design flaws fragility etc);
I'm reminded of my informal rule for live music: The quality of the band is inversely proportional to the number of sound checks they make.
Admin
Decision coverage doesn't test every single input to a condition.
What it does is test each possibility (true or false) for each decision, and shows that each input into that final "true or false" has an effect on the decision. Typically if you find an input to the Boolean expression has no effect on the Boolean it's an error of some sort. It also ensures that conditions which are supposed to be mutually exclusive are, in fact, exclusive.
This way you test all outcomes rather than all possible ways to get that outcome. Of course, this testing is generally used in safety-critical applications where the main goal is safe operation, not some specific functionality; the goal of this testing is to ensure safe states due to (or at least gain a full awareness of the effects of) all decision paths.
Admin
Congratulations, you've found the limit! You see, pendantry can also work against you.
Admin
So while the clock might be 99.999% correct, it's that 0.001 that causes an error and leads to a failure of the equipment tracking an incoming Scud missile to hit the target... which lead to the death of 28 service members. Look at http://www.ima.umn.edu/~arnold/disasters/patriot.html for more information.
Admin
next time i'll just leave my snarky comments to myself ... i'm out of orange crayons, and besides they're obviously not appreciated
Admin
Admin
Clearly QA people are not monkeys. They are baboons: http://www.newtechusa.com/ppi/talent.asp
Admin
I have some problems with that theory. The main one being that it is crap...
http://cache3.asset-cache.net/xc/3176763.jpg?v=1&c=IWSAsset&k=2&d=77BFBA49EF878921F7C3FC3F69D929FDC3F7A0BDA4B902D54AE2144BAD5428282D78032B8FE88392A7CFF610D5B4FC25
Admin
If you treat testing as a sort of janotorial function, one which I suppose we need but the people we hire to do it aren't really people, not like you and me, we can get it done for cheap by hiring a few interns from the local community college and making the low coder on the totem pole supervise them - well, in that case you get what you deserve, and that's dribbling idiots who can follow a script and that's about it.
Same with documentation - if you hire the lowest-cost warm body who can possibly be expected to do the job, you can maybe hope for something that's not very embarrassing.
Imagine if you hired coders that way - oh, wait. Some people do, and that's why this site exists.
Yeah, if you assume that testing doesn't matter, this is what you'll end up with.
(by the way, IQ 100 is by definition normal - calling people with definitionally normal IQ is, I suppose as clearly elitist as you could get and still be a commoner)
Admin
Refactoring has a much larger change impact, since it typically touches so much more of the codebase. While my customers certainly don't mind paying shit tons extra to risk breaking their already-working software for no functional benefit, I'm just not too keen on all the extra testing I'd have to do.
Admin
Admin
Ah, great a testing article written by a developer. (I kid, it's a good article, but I do want to leave a few notes, if you don't mind)
Testing is done to ensure that the application that's built adheres to the design specifications and business requirements. That doesn't mean fully bug free. So indeed, 'adequate' not 'complete'.
Someone designs the application, preferably using a standard like UML or something like that, and have the team review it.
Then someone else develops the application, based on said designs.
And you get someone else to test said application, with scripts based on the initial documentation, scripts that are reviewed by the team working on the project, written with a standard methodology, like ISTQB or TMAP Next, so that you can ensure the quality of the tests themselves.
Using this set up, you can be reasonably sure, based on the depth of the used test-techniques, and the used testdata, that there won't be any show stopping bugs (in the specified functionality), and perhaps only minor bugs (again only in the known specified functionality). And with the reviews you'll have closed the "who guards the guardians" gap as well.
This takes time, money and effort, and not every project has that in abundance (more often than not it lacks the time and resources to fully commit to it, but then you appoint key stakeholders that make time to do a basic review)
And you can then choose to only really put a lot of effort on the stuff that's critical, those priorities are set by the client, not a developer or a tester, but a client.
No risk, very little (or no) test.
On the flipside: no documentation, also no test. If a developer creates a lot of extra (perhaps handy) stuff, but it isn't designed in the specs, how is the tester to know what that functionality does? So, often the choice is made not to test that (or only very rudimentary test, does it do something that isn't terribly bad like lock stuff up?). Unless design specs are written up, and it can be incorporated into the test. Then it will be tested more thoroughly.
A few extra (and perhaps personal) notes:
For me as a tester, AGILE development schemes doesn't make a lot of sense, as it can cause a lot of headache, trying to keep track off the changes in the documentation. Moving targets aren't fun if you have to make hundreds of testcases, and the specs change by the day. Especially since testings is almost always done on the critical path, it often makes a very tight rope to do the foxtrot on.
And you NEVER let a developer test his own stuff, because it'll be hard get a grip on the quality in that case. No one is truly really critical on his or her own creations. "Oh, I know what I mean there. I'll just leave it in." (also testers shouldn't review their own testcases, let someone else do that)
Admin
Arghh the sad depressing sight of developers not getting testing.
Testing is "A technical investigation done to expose quality-related information about the product under test" (Kem Caner http://kaner.com/)
What most of you think of testing, is in fact just checking. You don't write unit tests, you are writing unit checks. Thinking about what you need to check, which code paths are relevant, which are not is the testing bit. Writing unit checks is just another thing devs need to learn. (for a better & longer explanation http://www.developsense.com/blog/2009/08/testing-vs-checking/ )
Far too many developers are just not intellectually suited to doing testing, just as testers are not suited to doing development. That does not mean it is easy.
I work very closely with my devs, I review their "unit tests", they help me with my automation when needed. Doing this we eliminate a huge chunk of defects at design time. This frees up my time to do the exploratory testing that finds the bug that happen only on certain pre-conditions that would never be found by code coverage.
Admin
That's why you can use a checklist instead of test cases. Or, gasp, carry out exploratory testing. Or use BDD. Etc, etc. Did the entire context driven movement pass you by?
Admin
Plato is first credited with asking the question in his "The Republic", but the Latin is definitely Juvenal.
Admin
Unit Testing is the process of testing a very specific bit of code. Proper unit testing involves mocking out database or file access and testing the operation of a specific class or function.
Integration Testing tests the full (integrated!) stack of code, such as you might find in a normal runtime. Integration tests will write to the database or filesystem, and as such they are expected to take more time to run than unit tests.
Unfortunately, there is much confusion surrounding this issue, even among developers. Perhaps this is due to the fact that testing frameworks often have the word "Unit" in their name, in spite of the fact they can be usually applied to any sort of automated testing.
Admin
Because, quite frequently, tests are a good place to start when reading the code, and a good way to specify, or at least closely track, whether the code actually conforms to the specifications.
Then, when things go wrong, you have the test framework as an entire other way to figure out what's going on.
Also because doing it this way (particularly with unit tests) forces you to design for testability, and such a design has other benefits. Generally, when you need to fix a bug (or "correct a defect," if you like), it's extremely helpful to be able to easily isolate the problem into one small test case. Even if you haven't written the particular test that should fail in this case, it's a lot easier to do if your code has been forced to be testable. And of course, if you do write a failing test, you now have a regression test for when you actually fix it, and a useful tool for the process of fixing it.
I agree that the real goals should be, well, the real goals. If the choice is between 100% test coverage and being able to deploy to production without breaking things, that's a no-brainer -- tests are a tool to make your program work, but clearly the goal is to make your program work. I just wanted to point out why, if you do suddenly find a really serious defect (like not being able to deploy to production), generally, the more comprehensive your test suite, the better off you are.
Also:
I guess that's TRWTF, because this makes me weep for my profession. It's not really a problem either if the parishioners need to duck because the door to the church is too low, but I would fix it anyway out of sheer embarrassment -- how hard is it to get this right?
I understand that Father Cronin doesn't need five nines for his database, and maybe we don't have to care about Unicode support, but there's a difference between a tricky-but-obscure defect that's not worth fixing and having some pride in your craft.
Admin
Admin
My bad, I didn't realize 100 was normal (I thought 110 or 115 was), so you can keep your vitriol...
Either way, I'm not disagreeing with you or the whole concept of testing. It is very necessary, however what I was attempting to illustrate is that it is often grossly underrated by management and other key decision makers. This results in conditions outlined in my rant (mouth-breathers going through 'Test Case 43, Steps 1-8'). They are usually only slightly more tech savvy (i.e. setup router) than your average person.
Admin
Alex's Soapbox was what made The Daily WTF a keeper. Sadly, we don't get enough of these insightful articles.
Keep 'em coming, Alex?
Admin
Depending on where you are, it may be the organization itself that prevents the testers from acting like they know what they're doing.
As a former tester, I've been on both sides of the equation, and I have been in situations where the lead tester demanded that we stick to the test script, icebergs (and crashes!) be damned, and as a coder, I have talked to the testers and found out, surprise, surprise, many of them have a pretty good idea of what's happening under the hood (even if only from a layperson's perspective), but alas, they've been instructed, on pain of death, to stick to the script and keep their bug reports non-technical.
I came from the game industry, which is frequently a massive worse-than-failure unto itself and goes out of its way to perpetuate animosity between the engineers and the testing staff, so things may be different.
Admin
That was in regard to the 'qa monkey' comment.
Admin
No, the distribution of IQ is normal. 100 is the expected value.
I can't be a commoner... not when I'm a citizen of imperialist America.
Admin
How about the REAL WTF here. A company I worked at let all of its testers go, leaving only the actual devs to test. When you ask a dev to test (self included), we have "blinders" on for the list of possible ways to break something...And to this day there is no dedicated test dept. In an IT dept of 40-50 devs, not mom & pop and the chimp :)
Admin
I chose that example because it's relatively easy to forget to add .Replace("\n", "
") when outputting HTML, and it's something I'm sure we've all done. While that example is trivial to fix, there are plenty of defects that are neither the builder's nor the client's fault, and that are not trivial to fix (let's say, a full week of time).
The onus is on the party that assumed the risk of the unknown. If it's a "fixed bid" agreement, then it's the professional obligation of the builder to fix and deliver the promised results. But if it's a "time and materials" agreement, the client has to pay to fix it. If they don't think it's worth 40 hours to fix it, even the most dedicated crafstman will not give up a full week just to have a better craft.
Admin
[quote user="C-Octothorpe"] My bad, I didn't realize 100 was normal (I thought 110 or 115 was), so you can keep your vitriol...
No vitriol intended. Just pointing out that there was a place where you were clearly being elitist. Unintentionally, as it turns out.
I agree with you on the symptoms, and it sounds like you agree with me on the cause. No fuss, no bother.
Admin
They call them hookers in these parts...
Anyway I thought this was the daily wtf, not slashdot.
Admin
What??? Did Alex become Jeff (Atwood) overnight?
Admin
Have a look at James Bach's stuff - there's a man who wants to test, and appears to be quite good at it.
Admin
And apparently, you also died and coped with alcoholism, at the same time.
Admin
I recently came across one where a database was happy to store the string "\0\0\0test" when almost nothing else in the environment saw it as anything but an empty string. Good luck writing a unit test for that one before you encounter it...
Admin
I did testing for Xbox 360 games for a year. I loved it, absolutely loved it, and the developers loved me because (since I loved it) I found a ton more bugs than the drooling morons who are usually in that position. And I daresay I'm pretty goddamned good at it.
The problem is that there's no career in it. The only way to make a career in QA is to move up into management, where instead of actually testing the software, you do nothing but herd the drooling morons around.
If you're a developer, you generally can work an entire career while still developing software; that doesn't exist in QA.
Admin
No shit. We have about 450 build targets in the automated build system. About a dozen are currently broken. And often it's obvious the code couldn't compile. (In theory, everything is supposed to be peer-reviewed too)
Admin
100% confidence is not 100% certainty. You could be 100% confident and 100% deluded.
Admin
No, see, his boss is called "Badly". Like "Joe", or "Ed".
Admin
So, a = b therefore c + b = d?
You see, pendantry can also work against you.
Admin
I thought the wtf was the fridge being too short. That small amount of useless space would drive me crazy.
Admin
Your pretty easily driven crazy, aren't you?
Admin
Your pretty easily driven crazy, aren't you?
Admin
Your pretty easily driven crazy, aren't you?
Admin
Your pretty easily driven crazy, aren't you?
Admin
let's watch and see what happens...
Admin
Admin
So, if A = 99 and B = 0.999..., then A + B = 99.999... (the original). Since 0.999... = 1, B = 1, thus 99.999... = A + B = 99 + 1 = 100
Admin
Thanks, C-Octo. I owe you a beer. (I love to see them pop like that)
Admin
And for the record, I wasn't being pedantic .. I was being snarky ... and now I'm going to be sardonic ... it's "pedantry" not "pendantry"
Admin
Wrong kind of confidence.
Admin
You still run the risk of an airplane crashing through your roof, or heck, a bus running through your front door. Maybe Nagesh and his taxi will let themselves in.
Admin
I have tested your software. Pray I do not test it any further.