• bvs23bkv33 (unregistered)

    unfristed comment

  • King (unregistered)

    Assert.IsTrue(date == "26/08/2016");

    No?

  • Murray Linwoord (unregistered)

    I don't see the WTF, this is how I always write my tests.

  • P (unregistered)

    We can literally write a new TDWTF article every day for every misuse of test-driven development, because they exist everywhere. There was another example that I found today, which goes on to show how prominent it is.

    Too many people think that TDD is just "write tests, give a list of pairs of input and expected output, and done" or "mock everything out", which is very, very far from the intention of TDD. TDD is not about the existence of tests; it's about two things:

    1. designing helpful, useful tests that will help you verify parts of your code are correct conveniently and quickly
    2. designing your program so that you can test your code conveniently and quickly
    3. (optional) automate this process

    Everything else, including mocking, test assertions, CI hooks and integrations... are just the means to achieve this goal.

    Doing TDD like how most people do it is kinda like taking a sip on every bottle of wine in a batch to ensure none of them are spoilt: completely missing the point, and causing more harm than its worth.

  • Naomi (unregistered) in reply to Murray Linwoord

    A developer checked in a test that asserted standard library functionality - useless but harmless on its own.

    In this case, the test depended on DateTime.Now representing a specific date (presumably the date the test was checked in), so the test isn't just useless - it's a time bomb set to go off during the next day's build.

    When the test failed, the response should have been to investigate it, delete the bogus test, and talk with Murray about why they thought it was a good idea (and the explanation may have been as simple as "I was debugging a bizarre bug, wrote some temporary tests to rule out some edge cases, and neglected to remove one of them before committing"). Instead, they turned off tests in their CI pipeline and left it that way for years. That suggests something is severely WTF about their corporate policies and culture.

    Put it another way: if a kid drops a jelly sandwich in your living room, you mop up the sandwich and tell the kid to be more careful. This company's reaction is more akin to cordoning off the living room.

    (There's also another explanation, which is that tests were disabled from the get-go, even before Murray's bogus test got committed. In that case, Murray's test is still a WTF, and the corporate policy is also still a WTF, just a different one.)

  • Naomi (unregistered)

    Aaaaaand now I notice the username I was responding to. Bravo zulu, you got me. :P

  • Tezra (unregistered)

    It would be bad enough if they were just testing the Date library, but they are actually testing that assignment still works.

  • Beliar (unregistered) in reply to Tezra

    Actually, they are testing that Assert.IsTrue accepts a string and somehow verifies if it is True.

  • LCrawford (unregistered) in reply to King

    Assert.IsTrue(date == "26/08/2016");

    No?

    No, it should have been Assert.IsTrue(date == "08/26/2016"); (there are not 26 months in the year)

  • Guest (unregistered)

    So where is the explanation as to why the build server didn't automatically run tests?

  • (nodebb)

    It's an invalid unit test because it tests 1) .NET BCL, 2) the unit testing framework, 3) Windows time management, 4) internet time provider, 5) their own awareness of today's date. As such, this becomes an integration test and not a unit test.

  • Anon (unregistered) in reply to Beliar

    I had assumed this was C#. In which case I can't see how it compiles, unless it's using a testing framework I am unfamiliar with, where the IsTrue method can accept a string.

    Because isn't the result of an assignment just the string being assigned? And there is no implicit string to bool conversion in C#.

    So Assert.IsTrue(d = "28/6/2016") shouldn't even compile.

    So...maybe none of the test project is compiled at all. Or I'm wrong about C#.

  • Beliar (unregistered) in reply to Anon

    Exactly, this test only compiles if Assert.IsTrue accepts a string.

  • masonwheeler (github)

    Now, this test could work if you mocked out the DateTime object, which is what you should do if you're testing dates. Of course, that doesn't make the test any more useful, as it's just testing basic, built-in .NET functionality.

    As is so often the case, tests involving mocks end up being more about the behavior of the mock than about the behavior of the actual code you're supposed to be testing.

  • (nodebb) in reply to masonwheeler

    That generally happens when people sit down to write unit test without first having any idea of what they are trying to accomplish. The mock shouldn't even have a "behavior", it should simply be the source of chaos that needs to exist in order to write code that answers the question "So, what does our code do when this happens?".

    Next, some idiot will pipe up and say "don't worry, that will never happen". How his comment is received will tell you whether your team is going to be one that writes robust code (the team rolls their eyes at the commenter) or one that constantly tracks down "mystery bugs" and blames other people (the team gets behind the commenter).

  • I dunno LOL ¯\(°_o)/¯ (unregistered) in reply to Mr. TA
    1. the locally configured date format... ISO-8601, folks!
  • I dunno LOL ¯\(°_o)/¯ (unregistered)

    ...and again, Markdown is TRWTF, thanks Jeff!

  • Angela Anuszewski (google) in reply to Naomi

    I wonder how many understood "Bravo Zulu".

  • (nodebb) in reply to Angela Anuszewski

    I didn't.

  • Colin (unregistered)

    "was automatically pulled by the build server" vs "the build server didn't automatically run tests." Some magic then?

  • Colin (unregistered) in reply to emurphy

    Me neither. Is it like Foxtrot Oscar?

  • (nodebb)

    Nope...

    It is Foxtrot Uniform Bravo Alpha Romeo.

  • Calli Arcale (unregistered) in reply to Colin

    Bravo Zulu basically means "good job".

  • Dlareg (unregistered) in reply to Beliar

    Do not know which language but in C assignments return true success.

  • (nodebb) in reply to Dlareg

    in C assignments return the value that is being assigned. So that a = b = c sets a and b to c.

  • (nodebb) in reply to Jaime

    "don't worry, that will never happen" - The one true way to ensure that something DOES happen.

  • (nodebb)

    I still can't see why this would cause all of the other tests to not be run? It's been ages since I've used anything other than the integrated unit testing framework in .NET, but a sensibly configured CI pipeline would execute the entire test suite, despite this one failing, however the attribute syntax is different from the built-in unit testing framework which would use [TestMethod] rather than simply [Test]. Anyone know what the go is with this?

  • (nodebb) in reply to Jaime

    The mock shouldn't even have a "behavior", it should simply be the source of chaos that needs to exist in order to write code that answers the question

    Please don't tell me you're an advocate of non-repeatable tests. I worked with someone who loved to do this, and when the code correctly threw exceptions for invalid input, the test would fail. If you're going to use any source of entropy or "chaos" at least log the seed so any failures can be reproduced.

  • (nodebb) in reply to TheCPUWizard

    "don't worry, that will never happen"

    Most people seem to only test for the "success" case, when the program operates correctly on valid input data. Almost nobody tests that the program will throw an exception on invalid data.

    One of my favorite ways of breaking other people's code is to pass it a collection that contains null somewhere in the middle of it.

  • (nodebb) in reply to King

    Assert.IsTrue(date == "26/08/2016");

    There's nothing wrong with this test, it's just incomplete. First it needs to set the system date to be August 26th 2016.

    Just add the following to the top of the file,

    [DllImport("kernel32.dll", SetLastError = true)] public static extern bool SetSystemTime(ref SYSTEMTIME st);

    Store the current Date / Time Start a Stopwatch (Just in case your test spans midnight) Call SetSystemTime, setting the date to be August 26th 2016 Add the elapsed time from the stopwatch to the previously stored DateTime and call SetSystemTime again with the calculated current time. It's foolproof. And a great way to "Mock the clock"

  • (nodebb) in reply to Jaime

    Next, some idiot will pipe up and say "don't worry, that will never happen".

    And I will tell him the tale of a system where there was a window three assembly instructions long during which something (a specific interrupt) could happen that would cause a memory corruption. My colleague had been hunting this corruption for a week or so, perhaps longer, and then I happened to look at a piece of code that we shared, and saw that an interrupt that occurred there in the code would cause the (non-paranoid) handler to proceed with the wrong value in DS:, and (digs) oh, look, that would remap these variables to be on top of that string in memory.

    Which string? Why, of course, the one that was being corrupted.

    He dismissed it as catastrophically improbable, to which I responded that until he fixed what I had found, there's no way he'd be able to find the real cause. He grumbled and fixed it, and the problem went away, never to return.

    (("non-paranoid interrupt handler"? Well, it didn't save DS and set it to be the correct value regardless, since it wasn't supposed to ever be wrong. During a later port to a different compiler, I was unable to preserve that assumption, so the interrupt handlers all became paranoid.))

  • D (unregistered)

    On this note, I don't suppose I'm the only one to spend time this week fixing automated tests that fail in the first few days of a new year due to the inability of the developers to deal with out-by-one errors, or do basic math?

  • (nodebb) in reply to idzy

    Please don't tell me you're an advocate of non-repeatable tests.

    Nope. By "chaos", I meant carefully chosen data that induces chaos in our code. Things like null, empty collections, negative numbers, and other edge cases that are a common source of bugs.

  • (nodebb)

    No WTF as I guess it tests:

    (a) that mocking works, probably in the test setup, and/or (b) that the date format string is the (stupid) M/D/Y format which might be part of the spec.

    /Jon

  • (nodebb)

    Joda time has a special method to override the current date. So you can write tests without updating them daily.

    When that method call ends up in your production code, you're in for a long debugging session.

  • Klint-psk (unregistered) in reply to LCrawford

    Spotted the "american who thinks the entire world functions as his country". The "Month/Day/Year" order is stupid, honestly. It must be either "Year/Month/Day" or "Day/Month/Year".

  • (nodebb) in reply to D

    I don't suppose I'm the only one to spend time this week fixing automated tests that fail in the first few days of a new year

    From 00:01 to 03:41 on 1st January, I got a new alert every five minutes for something that had actually happened a little after 8pm the previous evening. I don't know the details, but it seems pretty likely that someone left out the year from their "did something happen in the last n minutes" check.

  • doubting_poster (unregistered)

    The wtf is the assignment in the assert, clearly. The test itself is not a wtf. It's a test to make sure you have your test environment setup correctly and the date source is a mocked object returning the given date. That's fine. If you're not using the mocked date source, likely many hundreds of other tests would fall over with misleading errors.

Leave a comment on “Untested Builds”

Log In or post as a guest

Replying to comment #:

« Return to Article