Classic WTF: The Bug That Shut Down Computers World-Wide

« Return to Article
  • jmroth 2009-01-01 00:33
    Fist of teh year
  • ckd 2009-01-01 00:50
    Apparently all the old Wang VS programmers went on to the Zune software team.
  • ITEric 2009-01-01 01:20
    WOW! I don't think I've even SEEN a Wang since then. Was that what happened to them then?
  • Andy Goth 2009-01-01 02:29
    And we've been living on borrowed time ever since.
  • merichar 2009-01-01 02:36
    > Specifically, it mistakenly incremented the zero-indexed day-of-year
    > for all day-of-years that are equal to 0 in a leap-year.

    I may be misinterpreting the bug, but if it only incremented day-of-year 0 of a leap year, e.g. 1/1/1984, and did not increment any other day-of-year, couldn't this problem have been temporarily solved by waiting until day-of-year 1, e.g. 1/2/1984?

    On 1/2/1984 the temporary files generated would have expiration dates equal to the current date and the temporary files generated on 1/1/1984, with their expiration dates of 1/2/1984, could also be removed without raising an error.

    I suppose I'm just wondering why they sprung for overnight shipping, getting it there on the 2nd, when they had four years to send out the patch.
  • lolwtf 2009-01-01 03:01
    Hehe, wang.

    Whatever became of this programming language "so simple a secretary could master it"?
  • Kender 2009-01-01 03:29
    Here's the Zune bug:

    BOOL ConvertDays(UINT32 days, SYSTEMTIME* lpTime)
    {
    int dayofweek, month, year;
    UINT8 *month_tab;

    //Calculate current day of the week
    dayofweek = GetDayOfWeek(days);

    year = ORIGINYEAR;

    while (days > 365)
    {
    if (IsLeapYear(year))
    {
    if (days > 366)
    {
    days -= 366;
    year += 1;
    }
    }
    else
    {
    days -= 365;
    year += 1;
    }
    }

    http://pastie.org/349916
  • Wang 2009-01-01 05:11
    No post about Wangs is complete without a link to Freezepop's song - Do you like my Wang
  • Patrick Lin 2009-01-01 06:07
    Ironic that the Zune bug is also because of a programming oversight on a leap year.
  • configurator 2009-01-01 06:29
    So the zune bug would cause an infinite loop when days is 366 on a leap year?
  • m0ffx 2009-01-01 07:25
    merichar:
    > Specifically, it mistakenly incremented the zero-indexed day-of-year
    > for all day-of-years that are equal to 0 in a leap-year.

    I may be misinterpreting the bug, but if it only incremented day-of-year 0 of a leap year, e.g. 1/1/1984, and did not increment any other day-of-year, couldn't this problem have been temporarily solved by waiting until day-of-year 1, e.g. 1/2/1984?

    On 1/2/1984 the temporary files generated would have expiration dates equal to the current date and the temporary files generated on 1/1/1984, with their expiration dates of 1/2/1984, could also be removed without raising an error.

    I suppose I'm just wondering why they sprung for overnight shipping, getting it there on the 2nd, when they had four years to send out the patch.

    The bug had flooded the storage with dud temporary files. The wangs were unable to boot with no storage. Presumably they had to overnight some means to free it up, say a deletion program that ran bare-metal and didn't try to write anything (or even extra storage).
  • Shmork 2009-01-01 08:54
    What strikes me as odd is that for all of these date-related problems, nobody ever tried setting the system date ahead or to a leap year or anything else. But I guess until you see something fail spectacularly, you don't worry about it much. Though I wonder if the Zune fiasco will change that—that's some pretty awful publicity.
  • Jamesface 2009-01-01 09:01
  • Lupus.Umbrae 2009-01-01 09:05
    There were three big rages back in the 1980’s: Rubik’s cubes, Molly Ringwald, and Wangs. While I’m sure you are all familiar with those(...)


    So, who the hell is/was Molly Ringwald?





    Also, happy new year to everyone!
  • Someone Important 2009-01-01 09:25
    Shmork:
    What strikes me as odd is that for all of these date-related problems, nobody ever tried setting the system date ahead or to a leap year or anything else. But I guess until you see something fail spectacularly, you don't worry about it much. Though I wonder if the Zune fiasco will change that—that's some pretty awful publicity.


    Well, if 'it works', why test it further? After all, time == money...
  • sluq 2009-01-01 09:46
    "grinded"?
  • pjt33 2009-01-01 10:07
    Why would the temp file module need to do any arithmetic in the first place? Just set the file expiration time to now() and you're sorted. The impression I get from the article is that it was taking now(), converting it to YYYY-MM-DDTHH:MM:SS and then converting it back. Is that correct?!
  • Nunja Business 2009-01-01 10:12
    sluq:
    "grinded"?


    Thank you.

    Captcha: secundum - when just one trojan isn't enough?
  • Daniel Beardsmore 2009-01-01 11:21
    Lupus.Umbrae:
    There were three big rages back in the 1980’s: Rubik’s cubes, Molly Ringwald, and Wangs. While I’m sure you are all familiar with those(...)


    So, who the hell is/was Molly Ringwald?





    Also, happy new year to everyone!

    You know, looks like you answered your own question there.
  • ATimson 2009-01-01 12:04
    Shmork:
    What strikes me as odd is that for all of these date-related problems, nobody ever tried setting the system date ahead or to a leap year or anything else.

    The article makes it sound like it's only a problem on 1 January <leapyear>. So while they probably tried and tested against the leap day proper, and the days around it, they wouldn't have noticed anything on those other 365 days.
  • Steeldragon 2009-01-01 13:25
    Interesting WTF.
  • Crash Magnet 2009-01-01 13:27
    Someone Important:
    Shmork:
    What strikes me as odd is that for all of these date-related problems, nobody ever tried setting the system date ahead or to a leap year or anything else. But I guess until you see something fail spectacularly, you don't worry about it much. Though I wonder if the Zune fiasco will change that—that's some pretty awful publicity.


    Well, if 'it works', why test it further? After all, time == money...


    and

    knowledge == power...
  • dkf 2009-01-01 14:47
    Crash Magnet:
    knowledge == power
    That omits a few important steps in between...

    knowledge == academia
    academia == gas-bag
    gas-bag == wind
    wind == electrical generation
    electrical generation == power

    See? It's obvious!
  • Dirk Diggler 2009-01-01 14:51
    Crash Magnet:
    Someone Important:
    Shmork:
    What strikes me as odd is that for all of these date-related problems, nobody ever tried setting the system date ahead or to a leap year or anything else. But I guess until you see something fail spectacularly, you don't worry about it much. Though I wonder if the Zune fiasco will change that—that's some pretty awful publicity.


    Well, if 'it works', why test it further? After all, time == money...


    and

    knowledge == power...
    Sorry can't help myself.

    Chung Mee: Opium is my business. The bridge mean more traffic. More traffic mean more money. More money mean more power.
    Lawrence Bourne III: Yeah, well, before I commit any of that to memory, would there be anything in this for me?
    Chung Mee: Speed is important in business. Time is money.
    Lawrence Bourne III: You said opium was money.
    Chung Mee: Money is Money.
    Lawrence Bourne III: Well then, what is time again?
  • WTF 2009-01-01 15:04
    Kender:
    Here's the Zune bug:

    BOOL ConvertDays(UINT32 days, SYSTEMTIME* lpTime)
    {
    int dayofweek, month, year;
    UINT8 *month_tab;

    //Calculate current day of the week
    dayofweek = GetDayOfWeek(days);

    year = ORIGINYEAR;

    while (days > 365)
    {
    if (IsLeapYear(year))
    {
    if (days > 366)
    {
    days -= 366;
    year += 1;
    }
    }
    else
    {
    days -= 365;
    year += 1;
    }
    }

    http://pastie.org/349916

    Now why the heck would you put that inside of a while loop? That's just begging for a screw-up and an infinite loop. Wow.
  • John MAosn 2009-01-01 15:08
    Wow, I would have to say that is one crazy bug!

    jess
    www.privacy.es.tc
  • Dan 2009-01-01 21:08
    ckd:
    Apparently all the old Wang VS programmers went on to the Zune software team.


    Actually, the buggy code that locked up the Zunes is from Freescale. (It's part of their clock driver) I can only assume that there are other devices out there suffering from a similar problem.
  • Harry 2009-01-01 21:26
    It is made by Microsoft, you know ;)
  • jimlangrunner 2009-01-01 22:11
    Kender:
    Here's the Zune bug:

    BOOL ConvertDays(UINT32 days, SYSTEMTIME* lpTime)
    {
    int dayofweek, month, year;
    UINT8 *month_tab;

    //Calculate current day of the week
    dayofweek = GetDayOfWeek(days);

    year = ORIGINYEAR;

    while (days > 365)
    {
    if (IsLeapYear(year))
    {
    if (days > 366)
    {
    days -= 366;
    year += 1;
    }
    }
    else
    {
    days -= 365;
    year += 1;
    }
    }

    http://pastie.org/349916

    Had to run that through my "code review module" a couple of times. (Hey - it's New Year's Day.) "1" and "60" work just fine. What could go wrong? Then I plugged in "365" and saw day 0 pop out. Ouch. I feel for the code review team that missed that.
  • guest 2009-01-01 23:59
    [quote user="jimlangrunner]Had to run that through my "code review module" a couple of times. (Hey - it's New Year's Day.) "1" and "60" work just fine. What could go wrong? Then I plugged in "365" and saw day 0 pop out. Ouch. I feel for the code review team that missed that.[/quote]

    I think the bigger problem is that IsLeapYear is true and yesterday was day 366.
  • Counsel 2009-01-02 00:08
    Dan:
    Actually, the buggy code that locked up the Zunes is from Freescale. (It's part of their clock driver) I can only assume that there are other devices out there suffering from a similar problem.
    No, look at Microsoft's comments at the top of the file. If Freescale didn't accept Microsoft's EULA, Freesoft isn't authorized to use Freescale's source code. Neither can any other device makers.
  • rast 2009-01-02 02:19
    Lupus.Umbrae:
    So, who the hell is/was Molly Ringwald?



    "Yes, you're a total fag."
  • Code reading is a skill 2009-01-02 02:38
    Reading the code, it appears the reason it's in a while loop is that "days" can continue accumulating indefinitely while the device is off.

    The while loop allows them to increase the number of years by the appropriate amount even if it has been off for a prolonged time, such as several years.

    I'm not seeing a much simpler way to implement this than this while loop. You can't exactly just use integer division here. I don't think using "while" is the WTF. It's just a bug.
  • Wyrd 2009-01-02 08:52
    please s/grinded/ground/ in above article.

    Ah, the Wang. Yeah, I've almost never heard of it before except in the Do It Yourself song:
    "Oh, IBM, DEC, and Honnywell, HP, DG, and Wang / Amdahl, NEC, and NCR, they don't know anything. They make big bucks for systems so they never want it known that you can build a mainframe from the things ya find at home."

    http://www.bsutton.com/Bill/Filk/diyAnnotated.txt

    --
    Furry cows moo and decompress.
  • Slackmaster K 2009-01-02 09:05
    Yesterday did not mark the anniversary of "October 2nd, 2007"
  • tgape 2009-01-02 09:54
    Shmork:
    What strikes me as odd is that for all of these date-related problems, nobody ever tried setting the system date ahead or to a leap year or anything else. But I guess until you see something fail spectacularly, you don't worry about it much. Though I wonder if the Zune fiasco will change that—that's some pretty awful publicity.


    Well, some of us *do* think to test our code with leap years. Some of us even go so far as to test our code with Dec 31, the year before a leap year, and the first and last day of every month in the leap year, and Jan 1 of the year after the leap year. Others of us go even more pedantic and write an automated test loop that will test every day in the leap year.

    Obviously, these bugs happen to people who don't do this.

    Of course, then there's my coworker who wrote an automated test loop that tested every day in 2000... Sigh. (And, yes, his code did have a leap year bug.)
  • tgape 2009-01-02 09:59
    Slackmaster K:
    Yesterday did not mark the anniversary of "October 2nd, 2007"


    It wasn't the anniversary of the story being posted - it was the anniversary of the story itself.
  • bramster 2009-01-02 10:11
    Worked on a Wang back then. . . Whenever new software was being acquired, the first question the software salesman would ask is "How big is your Wang?"


  • wintermute 2009-01-02 11:42
    So, who the hell is/was Molly Ringwald?


    Let me Google that for you...
  • z0iid 2009-01-02 12:03
    i like that. learned something new.
  • Code Slave 2009-01-02 12:52
    [quote user="guest"][quote user="jimlangrunner]Had to run that through my "code review module" a couple of times. (Hey - it's New Year's Day.) "1" and "60" work just fine. What could go wrong? Then I plugged in "365" and saw day 0 pop out. Ouch. I feel for the code review team that missed that.[/quote]

    I think the bigger problem is that IsLeapYear is true and yesterday was day 366.[/quote]

    Yes, 365 breaks out of the loop, but 366 doesn't, and on a leap year, there is no way to decrement days.

    I suppose the way to fix it is collapse the two 'if's into

    if (IsLeapYear(year)
    && (days > 366))

    Then again, they should be working today again anyway.
  • operagost 2009-01-02 13:01
    Dirk Diggler:

    Chung Mee: Opium is my business. The bridge mean more traffic. More traffic mean more money. More money mean more power.


    There were three big rages back in the 1980’s: Rubik’s cubes, Molly Ringwald, and Wangs.

    ... and Wang Chung! (Thanks, Diggler)

    P.S. everybody have fun tonight
  • Ray Tackett 2009-01-02 13:36
    I worked for Wang in those days -- on the VS from 1976 to 1987. The bug first occurred in 1976 on the LAST (366th) day of a leap year. It botched the file's expiration date such that you could not delete a file which had not "yet" expired.

    They swore it was fixed for 1980. It wasn't. It occurred again in 1984 and yet again at the end of 1988. Knowledgable system administrators ("system operators" back then) would set the date back to 30 December on the 31st, then reset it correctly on 1 January of the following year.

    Wang Laboratories was dissolved completely in the mid-90s.
  • hamBone 2009-01-02 13:42
    Once again we see the importance of boundary classes in testing. Testing every date in a specific year (or multiple years) is sometimes impossible, almost always unneccesary, and will surely kill ROI. It is much easier to find the end-points(New Year's) or known trouble spots (Feb 29), and then test n+1 and n-1.

    When you consider that this is a basic lesson one learns when they begin QA, and take into account how many headaches developers have been through with datesin general, it's really a no-brainer.
  • hamBone 2009-01-02 13:51
    It is much easier to find the end-points(New Year's) or known trouble spots (Feb 29), and then test n+1 and n-1.

    ...and n, of course. Sorry, I need to unit test my comments.
  • ChiefCrazyTalk 2009-01-02 13:51
    Can anyone say...Zune, and the Z2K9 bug?
  • Lupus.Umbrae 2009-01-02 15:11
    Well, next time I ask a completely stupid question such as "Who is Molly Ringwald?" I will use <irony> or <trollmode> tags, okay?
  • Andy 2009-01-02 15:22
    Shmork:
    What strikes me as odd is that for all of these date-related problems, nobody ever tried setting the system date ahead or to a leap year or anything else. But I guess until you see something fail spectacularly, you don't worry about it much. Though I wonder if the Zune fiasco will change that—that's some pretty awful publicity.


    We have been hit with some date related issues. Its not just leap year, we found some other magic dates over time that cause problems. Nothing like that early morning call that Tokyo started trading and the whole system is down and every customer is calling. Fix it now.

    I have suggested to our QA team that they run with the dates in one lab set 3 months ahead. Call it the future lab. That way, if we hit a strange date related problem we have three months before all the customers call us.

    No, they don't do it.
  • Capt. Obvious 2009-01-02 15:51
    Code Slave:
    I suppose the way to fix it is collapse the two 'if's into
    [code]
    if (IsLeapYear(year)

    && (days > 366))

    Well, that would have the effect of breaking in a non-infinite loop way in leap years. The calander would still go Dec. 29, Dec. 30, Jan. 1, Jan. 1 at end of a leap year/start of the next year.
  • BoredAtWork 2009-01-02 16:32
    Shmork:


    ...

    Of course, then there's my coworker who wrote an automated test loop that tested every day in 2000... Sigh. (And, yes, his code did have a leap year bug.)

    Am I missing something here? That sounds like a *good* way to catch a leap year bug.
  • rast 2009-01-02 16:59
    Lupus.Umbrae:
    Well, next time I ask a completely stupid question such as "Who is Molly Ringwald?" I will use <irony> or <trollmode> tags, okay?

    You were just being sarcastic?



    "Oh yes that's it. That's exactly it."
  • Worf 2009-01-02 17:07
    jimlangrunner:

    Had to run that through my "code review module" a couple of times. (Hey - it's New Year's Day.) "1" and "60" work just fine. What could go wrong? Then I plugged in "365" and saw day 0 pop out. Ouch. I feel for the code review team that missed that.


    Considering the Zune OS is really Windows CE, there's no excuse for such code at all. Especially since Windows CE exposes a full set of date-related computation functions for you. I've worked on RTC drivers, and the way to get things set up is to take the current SYSTEMTIME and either program it into the hardware directly (a lot of hardware chips do BCD day/date/month/year/hour/minute/second calculations internally), or if it's lamer and does a seconds-since-some-epoch, you convert the SYSTEMTIME to a FILETIME (64 bit monotonically increasing humber since Jan 1904 - 100ns increment interval), compute the difference to your chosen epoch FILETIME, then convert from that to seconds (a nice round number... 10 million I believe), and boom, you're done.

    The only reason in the world to test a date would be to ensure that it's within bounds, and it's just checking that day of week is 0-6 (or 1-7), the months have proper days (only reason to need a leap-year test, and that code is around), and the month is between 0-11 (or 1-12) before starting the computation. Anytime I have to start writing code with dates, there's no reason to not use OS provided functions. Small embedded systems probably don't have date functions, but then again, maybe that's an indication to try to rewrite your code to either not use dates at all, or push the date stuff to a more powerful processor with an OS and date functions.
  • Alexis de Torquemada 2009-01-03 00:49
    configurator:
    So the zune bug would cause an infinite loop when days is 366 on a leap year?


    And I thought Apple's address was 1 Infinite Loop :-D

    You'd really think a company would run some unit tests on their firmware before shipping a gazillion devices (they wish), but no...
  • ikdind 2009-01-03 01:55
    Ugh... bad memories of Zune2k9, which I just had to go through. When will we learn that leap years happen every 4 years and have 366 days?
  • ikdind 2009-01-03 01:59
    To be fair, the original firmware worked just fine and a lot of people fixed their Zunes by disconnecting the batteries and forcing it to reset to its original firmware. It was one of the firmware updates after they had shipped tens of units that caused the infinite loop.
  • julmu 2009-01-03 14:21
    ikdind:
    Ugh... bad memories of Zune2k9, which I just had to go through. When will we learn that leap years happen every 4 years and have 366 days?

    Leap years happen every 4 years if you have the Y2100 bug.
    Hint: http://en.wikipedia.org/wiki/Leap_year#Algorithm
  • hikari 2009-01-04 08:15
    Lupus.Umbrae:
    So, who the hell is/was Molly Ringwald?


    She is an actress and was, back in the 80s, a member of the "Brat Pack" along with Emilio Estevez, Anthony Michael Hall, Andrew McCarthy, Judd Nelson, Demi Moore, Ally Sheedy and some others I'm probably forgetting.

    She's the ginger haired one from "The Breakfast Club". She was also in "Not Another Teen Movie" as a cameo, the check-in attendant at the end, iirc.
  • Dirk Diggler 2009-01-04 12:21
    ikdind:
    Ugh... bad memories of Zune2k9, which I just had to go through. When will we learn that leap years happen every 4 years and have 366 days?
    Actually, they don't happen every 4 years.
  • rast 2009-01-05 01:51
    hikari:
    She's the ginger haired one from "The Breakfast Club".
  • ParkinT 2009-01-05 08:32
    If I recall, I had trouble getting my Wang back up in 1984!



    If only I knew there was someone else to blame.
  • Dirk Diggler 2009-01-05 08:44
    ParkinT:
    If I recall, I had trouble getting my Wang back up in 1984!



    If only I knew there was someone else to blame.

    In 1984 I had no problems with my Wang.
  • InsanityCubed 2009-01-05 09:37
    jimlangrunner:
    Kender:
    Here's the Zune bug:

    BOOL ConvertDays(UINT32 days, SYSTEMTIME* lpTime)
    {
    int dayofweek, month, year;
    UINT8 *month_tab;

    //Calculate current day of the week
    dayofweek = GetDayOfWeek(days);

    year = ORIGINYEAR;

    while (days > 365)
    {
    if (IsLeapYear(year))
    {
    if (days > 366)
    {
    days -= 366;
    year += 1;
    }
    }
    else
    {
    days -= 365;
    year += 1;
    }
    }

    http://pastie.org/349916

    Had to run that through my "code review module" a couple of times. (Hey - it's New Year's Day.) "1" and "60" work just fine. What could go wrong? Then I plugged in "365" and saw day 0 pop out. Ouch. I feel for the code review team that missed that.



    I think you need to double check your "code review module"

    while (days > 365) should return false and day 365 will return. Now try day 366 on a leap year and you will see a real problem.
  • LordVashtal 2009-01-05 10:34
    Wang VS, eh? Sounds like cockfighting to me...
  • Charles400 2009-01-05 15:26
    You can't beat a Wang for a good date.
  • airdrummer 2009-01-06 09:17
    i'd just started with a new co., and the proj. was a cobol fin.mgt system on a wang...the file sys. had 1 dir. level, but the dynamic linking made programming in that limited space easy:-)
  • rfsmit 2009-01-07 16:24
    sluq:
    "grinded"?

    Indeed, I remember making a similar comment the last time it came up. Which I could've sworn wasn't 25 years or a year and a half ago.
  • Grammar, please 2009-01-09 04:31
    Using "grinded" rather than "ground" makes it sound like you should break the banjos out.
  • sfsad 2009-01-21 23:55
  • huojia 2009-03-02 01:15
    &nbsp;&nbsp;
    货架racking.&nbsp;&nbsp; 仓储货架
    南京货架设备要产品包括:重型货架
    ,阁楼货架,超市Shelf,重量型货架,横梁式货架,驶入式货架... 仓储笼
    货架的形式与材料 · 立体仓库 · 货架厂
    物流规划设计的步骤与程序(货架的设计... 中国物流行业呈现三足鼎立抢市场 · 亚洲第三方物流的现状和发展 ... 托盘
    抽出式货架系列 · 重力式 货架/推入式货架钢托盘
    移动式货架 · 阁楼式货架 · 悬臂式货架
    系列 长件物料储存货架&nbsp;货架公司
  • Some dude 2011-09-21 10:26
    Lupus.Umbrae:
    There were three big rages back in the 1980’s: Rubik’s cubes, Molly Ringwald, and Wangs. While I’m sure you are all familiar with those(...)


    So, who the hell is/was Molly Ringwald?



    My thoughts were more like: What the hell is a Wang?
  • Omega 2012-11-11 17:53
    Some dude:
    Lupus.Umbrae:
    There were three big rages back in the 1980’s: Rubik’s cubes, Molly Ringwald, and Wangs. While I’m sure you are all familiar with those(...)


    So, who the hell is/was Molly Ringwald?



    My thoughts were more like: What the hell is a Wang?


    If you don't know what a Wang is, are you sure you're a dude? Where's your dignissim?