• Juvenal (unregistered)

    One way you can tell that "Quis custodiet ipsos custodes?" is my quote and not Plato's is that I, being Roman, spoke Latin, whereas Plato, being Greek, spoke Greek.

    In future please attribute my quotations correctly. Thank you.

  • foo (unregistered) in reply to blakeyrat
    blakeyrat:
    The problem is that there's no career in it. The only way to make a career in QA is to move up into management, where instead of actually testing the software, you do nothing but herd the drooling morons around.

    Yeah, which means that for those of us who are QA Devs, we get a manager who has no idea how to manage devs (or, one suspects, manage anything; he's barely coping at his job). I guess you herd drooling morons around by getting a bigger drooling moron to create rivers of drool which they follow.

  • (cs)

    The Sixties called. They want their clock back.

  • Johan Bergens (unregistered)

    I have a minor comment about who is responsible for deciding the level of quality vs qantity. The customer doesn't always have the knowledge to decide the level of quality, or rather understand the implications of errors. Or he/she needs to describe it in a general form that the developers can understand and work from.

    Saying you have small error in the compressor of a freezer is not enough, you need to know if the problem might cause the freezer to stop working and spill fluids on the floor which might be a big deal for the customer or if it only might cause the freezer to hold a somewhat higher temperature than specified.

    Clients usually wants a cheap system but with high quality. They might ask if you have tested the system and it worked, and as you wrote testing can be many things. This is where I think BDD might help, the customer can read some tests and see if the tests reflects the wanted quality. Things not tested (caused by removing tests to save time or not writing them at all) might break/not work as expected.

  • Andrew (unregistered)

    One problem I consistently see when people write about testing is the assumption that functional test are test scripts which are then performed by a tester manually. The assumption is often that only unit tests are automated.

    For many (probably most) software this is true and valid. However, if a piece of software is expected to live and be updated for years, automating the feature tests may be necessary.

    For a large application which has to ship in multiple languages, run on multiple platforms and undergo regular updates the cost of manually testing the app over time can far outstrip the cost of automating the feature tests.

  • Craig (unregistered)

    What, nobody picked up that "quis custodiet ipsos custodes" was Juvenal, not Plato?

  • Craig (unregistered) in reply to Craig

    Oh yes, I see Juvenal himself did :-D

  • Prism (unregistered) in reply to Jay
    Jay:
    neminem:
    Tim:
    In fact, since the number of code paths through the entire code base grows exponentially you have covered some vanishingly small percentage of the code paths.
    I'd argue anytime your input comes from the user or from another system you have no control over the output of, you've covered exactly 0%. A finite fraction of an infinite is, after all, precisely nothing. (We once found a major blocking bug in a process iff the text input to the process started with a lowercase 'p'. That was a fun one.)

    But I'm happy to work at a computer that distinguishes between devs and testers; we certainly are responsible for testing the effects of our own code before checking in changes (that would be your #1), but the "test team" consists of people that were hired for that purpose. I sort of thought it was like that everywhere (well, everywhere where the whole company isn't a couple devs.)

    There's a difference between "all possible code paths" and "all possible inputs".

    Pedantic digression: The number of possible inputs is not infinite. There are always limits on the maximum size of an integer or lenght of a string, etc. So while the number of possible inputs to most programs is huge, it is not infinite.

    Suppose I write (I'll use Java just because that's what I write the most, all due apologies to the VB and PHP folks):

    String addperiod(String s)
{
  if (s.length()==0)
    return s;
  else if (s.endsWith("."))
    return s;
  else
    return s+".";
}

    There are an extremely large number of possible inputs. But there are only three obvious code paths: empty string, string ending with period, and "other". There are at least two other not-quite-obvious code paths: s==null and s.length==maximum length for a string. Maybe there are some other special cases that would cause trouble. But for a test of this function to be thorough, we would not need to try "a", "b", "c", ... "z", "aa", "ab", ... etc for billions and billions of possible values.

    That said, where we regularly get burned on testing is when we don't consider some case that turns out to create a not-so-obvious code path. Like, our code screws up when the customer is named "Null" or crazy things like that.

    "So while the number of possible inputs to most programs is huge, it is not infinite."

    Your example is rather bland. How about throwing a 40 or so character RegEx into the mix and coming up with all the inputs "proving" it works not matter what?

    You can describe all the 'valids' but never all the invalids, and that is the essential problem.

  • Prism (unregistered) in reply to Mr. Orangutan
    Mr. Orangutan:
    VoiceOfSanity:
    So while the clock might be 99.999% correct, it's that 0.001 that causes an error...
    99.999% != 99.999...% 99.999... implies the repeating 9 ... as we all should know, 0.999... = 1 therefore, 99 + 1 = 100

    next time i'll just leave my snarky comments to myself ... i'm out of orange crayons, and besides they're obviously not appreciated

    "as we all should know, 0.999... = 1"

    Its starting to become accepted that that is not true.

    If 1/infinity==0 then it is possible to divide a small enough number and end up with nothing.

    I am on the side that 1/infinity==0.000..<infinity>..0001

    so that 0.999....+(1/infinity)==1

    seems obvious to me that the series 9/10+9/100+9/1000... will never give 1 by definition. Never.

  • Mchl (unregistered)
    Alex Papadimoulis:
    I doubt you would bat an eye if the Mars Rover team tested commands before sending them using a replica Mars Rover sitting on a pile of replica Mars rocks

    While it's not a common thing, they actually do that sometimes: http://en.wikipedia.org/wiki/Spirit_rover#Stuck_in_dusty_soil_with_poor_cohesion

  • MH (unregistered)

    Interesting article. One area of testing I'm interested in is absense testing- i.e. if the program is supposed to identify something, such as the number of widgets with defects, or in banking to return the exact transactions that are fraudlent, is it returning all of them or only a portion of them? Testing the output does not account for items not on the output.

Leave a comment on “Testing Done Right”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article