• (nodebb)

    Fristly: code coverage should be in the 100-200% range! Secnodly: don't forget to cover the test code with tests!

  • (nodebb)

    If you measure code coverage, then you will get code coverage- and nothing else.

    Yes but no. It all depends on how your code coverage tool measures coverage. Some actually track which lines of application code get called, which (aside from questions of the time it takes and related things) is what you really want. Others measure nonsense like "we called a method of the class, therefore the class is 100% covered", which is very, very far from what you want.

    But even then, it's not a sufficient measure, because of a problem related to combinatoric enumeration. If you have a function where all its if()/else branches are covered, you might not have exercised all the permutations of branches, i.e. you have covered all the A paths and all the B paths, but you did it by covering AABA and BBAB, and there's a bug that only happens for BABB, so it ends up being found by a customer.

  • (nodebb)

    I should also point out that at $JOB we have some unit tests for libraries, and in one very important sub-library (a sort of object database), the unit test collection includes things like "I asked to create an empty group of objects, and the group that the library created was, indeed, empty" being the entirety of the test of object groups.

    At the same time, another module (a stateful IPS that verifies network packets) has an extensive test suite of about 9000 tests and nonetheless still doesn't achieve 100% coverage (by the "which lines did we hit" measure).

Leave a comment on “Are You Mocking Me?”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article