• (nodebb)

    It's worse than that, in fact. It doesn't test the toString method, since it doesn't look at what's returned. At best, it tests that it obeys the core contract of the methods, i.e. that they return something versus a NULL...

    It doesn't even verify that toString() returns a string, ffs.

  • (nodebb)

    assert(FRIST==FRIST);

  • Sauron (unregistered)

    TRWTF is that the project management office didn't see it coming, and just kept piling projects on the lead dev, with no fallback plan.

  • eluvatar (unregistered)

    Whenever someone mentions a code coverage target I always point out that I can meet that target without providing any value and I can write super valuable tests without getting anywhere near that target. Way more important to test useful things, but it's not like management would know what that looked like if it hit them square in the face.

  • WTFGuy (unregistered)

    The fact the lead's departure percolated up to the C-level says either the whole company is real small, or the PMO was essentially powerless and the mismanagement and extreme under-resourcing and overpromising goes all the way to the top. Which situation, though a major-league meta-WTF in itself, is so very common in our industry as to elicit nothing but a "Ho humm, move along, nothing to see here." reaction from me and I suspect from most of us. Which in turn is the meta-meta-WTF to rule them all.

  • (nodebb)

    Gee I wonder why the dev bailed.

  • (nodebb)

    that one developer's tasks were on every critical path on their Gannt chart

    Well there's your problem: they should try using Gantt charts instead.

    This keeps their code coverage metric pretty high

    How? Surely your coverage metric is the percentage of your code that is exercised by the tests. Unless the constructor, toString() and equals() methods are very large, the percentage is surely going to be close to zero.

  • (nodebb)

    Unless the constructor, toString() and equals() methods are very large, the percentage is surely going to be close to zero.

    Sort of, except that calling the boilerplate generated methods(1) will create a chunk of pointless "coverage", which boosts the percentage without actually adding much.

    (1) As I noted above, the calls don't actually test those methods, since the returns are not checked for correctness, unless two new blank objects should compare equal.

  • Kleyguerth (github)

    Fully agree with the easy reader version. I rather use snippets to generate boilerplate than "magic". "Magic" works wonders, until it doesn't...

  • Dave Aronson (github)

    Mutation testing would have caught those absolutely useless "tests". Y'all feel free to spread this meme I made:

    https://twitter.com/DaveAronson/status/1595077882199834624/photo/1

    And if you don't yet know about mutation testing, you can pick one of these conference talks of mine on it, of various lengths (and therefore depths): https://www.youtube.com/playlist?list=PLMrm16n64Bub8urB-bsyMyHiNPMLG7FAS

  • MaxiTB (unregistered)

    Ah, useless unit tests so that management can check something on their imaginary success list. Never gets old.

  • (nodebb)

    I hope they don't take test coverage percentage to be the minimum %'age of tests that pass in order for the build to be accepted.

  • Adam Ingerman (github) in reply to prueg

    No, no, the percentage of passing tests is only allowed to drop by 5% at a time...

  • (nodebb)

    "as that one developer's tasks were on every critical path on their Gannt chart. " - Root cause identified.

  • (nodebb) in reply to prueg

    I hope they don't take test coverage percentage to be the minimum %'age of tests that pass in order for the build to be accepted.

    To me it looks like they set a minimum code coverage for acceptance, combined with unrealistic deadlines. Then the dev started playing the metrics to meet the deadline or at least miss it by less.

    Who could ever have seen it coming?

  • Urs (unregistered)

    I came across a big project in my career which claimed to have 80 % code coverage. When you looked at the tests, they had no asserts at all - they were just calling the methods to be tested. But the code coverage report showed indeed 80 %. At least the tests made sure that there were no exceptions - so we thought - except that almost all the methods had a try{} catch(Exception ex) {_logger.LogError(ex);} around them...

  • Airdrik (unregistered)

    Makes me suspicious about the unit test coverage requirement itself. We have unit test coverage requirements for our java projects, some of which use (or have used) lombok for boilerplate reduction. However the test coverage only measures source lines of code as actually checked into the repo, not generated code. Even if it were considering generated source code, lombok doesn't generate java source code, it generates java byte code (unless you go out of your way to generate source code using delombok).

    Then, assuming the test coverage is including the generated code, I would also expect that generated code should be small-ish (or at least not much larger) compared to the actual checked-in code, which leads me wonder how it is that they can get test coverage on generated code up to sufficient levels to pass the requirement without even covering any of the actual code.
    This leads me to conclude that either they do at least have tests which "cover" actual code (by which we can assume means the tests run the code without verifying anything useful happens), or the data model is so large compared to the rest of the logic that the amount of generated code is big enough that they can meet the coverage requirement with only "covering" the generated code. Or the test requirement is a lot looser than they made themselves believe.

  • Anonymous Teacher (unregistered)

    This is precisely my students do when I ask them to write Unit tests. They make lots of tests but miss the whole point and pepper their tests with assert(True) statements. Then they show me lots of running tests and all of them pass.

    One of my students even succeeded in writing lots of tests without running any of their own code at all.

Leave a comment on “Unit Test Coverage”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article