• Belcat (unregistered)

    I can just hear what the first customer would have said, had they not caught it...

  • sammy (unregistered)
    Upon hearing of this highly destructive bug, Jerry had the intern submit a bug report with the steps to reproduce. And rather than try to trace the code to what could've caused such a horrible meltdown, Jerry decided to follow the steps to reproduce the bug on his computer. The computer that he needed to perform his job. The computer that he'd just completely hosed by trying to reproduce the bug.

    This was originally a Polish joke.

    Guy goes into a doctor's office. "Doctor," he says, "it hurts when I do this," and pokes himself in the shoulder with his finger. Then he pokes himself in the knee, belly, cheek, and thigh, complaining that each successive poke of the finger hurts.

    Doctor says, "You're in software testing, aren't you?"

    Guy says, "Yeah! How'd you know that?"

    Doctor says, "Your finger is broken."

  • Carnildo (cs)

    I've had a bug like that. I wrote a function that took a list of files and filename patterns and deleted them, as part of the "uninstall old versions of our software" functionality. Problem is, I made a typo about three-quarters of the way through the list. That typo interacted with the way that MacOS specifies relative paths to mean "Okay, now delete the entire contents of the folder containing the folder that our software was installed in."

    Since old versions of the software were typically installed in the root of the hard drive, guess what was deleted?

  • ArchAngelQ (unregistered)

    Caught a bug exactly like this in the uninstall for a streaming media application I was doing QA for not even all that long ago. One version would clear the whole C: drive if you installed it in a subdirectory of it. It in fact deleted whatever directory was above the current install directory. Default install path?

    C:\Program Files<company name><app name> of course :P

  • Nanoda (unregistered)

    We had some similarly awesome code involving .NET registration.

    What was supposed to happen was every time the program was run, it would list it's .NET libraries, determine it's location in the registry, delete the entries, and register them again.

    One day development wasn't working so well, and lacking another test machine I RDPd to my home PC and ran it there, which didn't work any better.

    Turned out that a slightly different way of compiling our .NET code combined with a bug in the registration code, which ended up deleting the user's entire CLSID registry entry on start.

    I didn't remember writing it, but subversion said it was my fault, so karma levels were appropriate.

  • pitchingchris (cs)

    Very good reason to always be specific about which file(s) to delete and be cautious when using wildcards. more than likely if they would have use *.(theirExtension) they probably wouldn't have found any files at the root

  • shadowman (cs) in reply to pitchingchris

    User Account Control to the rescue?

  • obediah (cs)

    Wow it didn't take long for to cash in all the karma gained from the awesome bomb cam post. :(

    "lul, I deleted my OS files" jokes are as old as..well..OS files. Which is older than most of the people posting here. You could have at least used an example with some WTF code behind the machine wipe.

    Alright, tough love is over. Thanks for the daily respite and keep up the good work!

  • FredSaw (cs)

    This story is kind of fractured. What has the WTF got to do with Jerry's assembler background not preparing him for C++? And don't you mean it was AJ who hired the team of interns, and who reproduced the problem on the second computer?

  • Christophe (unregistered)

    ...And in another fun advertisement juxtaposition, next to this story are ads for two different disk eraser utilities. Hope they work better than the app in the story.

  • Anonymous (unregistered) in reply to Belcat

    AJ stands for Apple Jax, I just know it.

  • voidy (cs)

    We had a problem like that. One of my colleagues made a script to clear space from the c: drive of production webservers, stuff like crash dumps and temp files. It basically mounted folders over SMB, and removed the contents. A few of us started using it, but one night it seemed to remove everything from c:\windows, and more, completely hosing a production webserver. It turned out that he was in one of the folders at the commandline when trying to unmount c:, and so mounting c:\some\folder didn't do anything before doing a rem . . We're not allowed to use it anymore :(

    managed to just copy the files from another webserver as we still had smb access, so it wasn't too much of a disaster

  • boyd (unregistered)

    Where's the story here? A bug has been discovered during QA, investigated and fixed before it affected any customers. boring.

    I can't see a relation between jerry's c++/assembly skills and the story either.

    And btw the bug is not in the function deleting . but in the function that return c:\ as the temp directory.

  • daBowmore (unregistered)

    Brillant! It was recursive!

  • JJ (unregistered)

    I had the exact same experience at my company. Our uninstaller used to wipe out the entire computer. it wasn't to long after that happened that we started basking in the joy of VMs.

  • Aaron (unregistered)

    One thing I don't get is... if it did a del . on the root directory... wouldn't that have JUST deleted the root directory? That's really not as catastrophic as it sounds provided you have another computer (with similar OS) nearby.

    now if it did "deltree -Y"...that'd be a different story. :)

    I could be mistaken on the implementation, the article wasn't specific enough on whether it was literally a "del ." command, or just acted like one.

  • Vlad Patryshev (unregistered)

    I am sure this is ubiquitous, "automatic file cleaning operation" that by mistake starts with root directory.

  • Corporate Cog (unregistered)

    It's not fun to admit that I did something almost as bad.

    We had a product that registered more than one "active X" dlls. That created many problems (and the active X route was later abandoned). Some folks' machines were getting all hosed up because those dlls were failing to remove themselves from the registry even when uninstalling. I searched the registry and there were ~ 5K entries that needed to be removed to make a machine clean.

    So I made an NSIS script to remove them all. Some of the entries were much like others except not as long. While copy/pasting and ctrl backspacing, I went too far on one. The reg key was HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion.

    If that key is deleted, the machine will work almost normally (only one strange thing was reported) until rebooted. There is no way to reboot. The file system must be accessed by booting to some other drive.

    Luckily it was only used internally, and only 2 folks were hit.

  • Corporate Cog (unregistered)
    Anticipating that future changes might result in different file extensions going in the temp folder, AJ told the developer to "just delete *.*."

    YAGNI violated.

  • a nonee mouse (unregistered)

    First week of a new job...back in the DOS 3.3 days...

    SW Dev staff was using a set of unix utilities available from DOS (I'd say cygwin, but it probably wasn't). This was my first exposure to rm - what a great thing to those of us stuck in DOS!
    We were all developing on drives mapped to the local Netware server - because, back then, compiling over the network was significantly faster than compiling on the local hard disk. Security was non-existent in this three-developer company; we all had access to each others directory.

    So imagine my surprise when a finger slip: rm -rf <development directory>
    which I ignored, figuring it would, at worst, cause DOS to bitch about a bad command, took a long time to execute. Imagine my surprise when the other developers looked up and asked "Are you having problems with the network too?". Imagine my horror as it slowly dawned on me what was happening, and as I started pounding on ctrl-c trying to get the damned thing to stop what it was doing.

    It only took down the shop for two days.

  • Fnord (unregistered)

    Do some developers have a problem with the concept of a "temporary" file? If the file is temporary, you should delete it as soon as you have finished with it. Not leave it on disk. If you leave it on disk, for whatever reason, it isn't temporary is it?

    This is obviously a Windows application so it would seem that using the application data folder would have been a better solution. Or actually tracking the handles of each "temporary" file that was created and deleting it via. the handle instead of the shotgun approach used here. Even if the wildcard was correct, it's still a dumb method.

  • iMalc (unregistered)

    That reminds me, be very careful with Uninstalling "Sierra Utilities". More specifically - don't do it! It puts this little app just outside the folder it installed to. E.g. C:\Games
    Thus upon uninstalling it removes not just C:\Games\SomeGame, but it actually deletes everything in C:\Games.

    I wasn't pleased that I then had to reinstall all of my games!

  • ParkinT (cs) in reply to Nanoda
    Nanoda:
    We had some similarly awesome code involving .NET registration.

    What was supposed to happen was every time the program was run, it would list it's .NET libraries, determine it's location in the registry, delete the entries, and register them again.

    One day development wasn't working so well, and lacking another test machine I RDPd to my home PC and ran it there, which didn't work any better.

    Turned out that a slightly different way of compiling our .NET code combined with a bug in the registration code, which ended up deleting the user's entire CLSID registry entry on start.

    I didn't remember writing it, but subversion said it was my fault, so karma levels were appropriate.

    I don't trust that log. It is always wrong (especially when it points to me!).

    <g>
  • ogilmor (cs) in reply to Carnildo

    The Java 1.4.2.12 installation was like that I think. It happened when you converted to an admin install. Thankfully most people didn't save a lot of data to c:\ and the files there are protected or we would have had a bigger disaster (we were pushing out the DST change with a logon script).

  • Someone You Know (cs) in reply to Corporate Cog
    Corporate Cog:
    If that key is deleted, the machine will work almost normally (only one strange thing was reported) until rebooted. There is no way to reboot. The file system must be accessed by booting to some other drive.

    I'm curious to know what the one strange thing was.

  • Matt (unregistered)

    Now, I'm curious.

    This sounds suspiciously like a bug found in pools of Radiance: Ruins of Myth Drannor, created by Stormfront Studions and released by UbiSoft. More than a few people reported that the uninstalled busily wiped the drive that contained the game.

    Made things worse with the fact that the game could only be installed on c:\

  • snoofle (cs) in reply to Belcat

    When, oh when are people going to learn? Whether you're on Windows or *nix or whatever, when you've got some code to do a search-and-destroy type of operation, make sure you do something like this:

    FileWildCard="*.xyz"
    BasePath="something to calc the starting path"
    if  BasePath = "/" then 
       error in figuring out starting path
       return <operation failed>
    fi
    delete ${BasePath}/${FileWildCard}
    

    ... it looks redundant, but is trivial in execution time and it can save your butt!

  • poopdeville (cs) in reply to Corporate Cog
    Corporate Cog:
    Anticipating that future changes might result in different file extensions going in the temp folder, AJ told the developer to "just delete *.*."

    YAGNI violated.

    Eh, not really. YAGNI, as it is most often understood, is a best practice because Laziness is a Virtue. The pedantically "right" solution to this problem would be to use an easily updatable configuration file to change (either at compile or run time) which files are deleted. Implementing that would definitely be a YAGNI violation since you would spend far more time writing and maintaining the thing than it would save you. On the other hand, using delete . doesn't cost you anything over delete *.ext while still fulfilling the informal specification.

    This is still a WTF though. Obviously, care should be taken when deleting files. But I think The Real WTF is that the function returning the path wasn't properly unit tested.

  • B (unregistered) in reply to Aaron
    Aaron:
    One thing I don't get is... if it did a del *.* on the root directory... wouldn't that have JUST deleted the root directory?

    If you were running MS-DOS, that would have taken command.com and config.sys as POWs. No booting after that.

  • Jason (unregistered)

    I nearly became guilty of something similar.

    We just switched anti-virus solutions, so we needed a way to uninstall the old software remotely and without user interaction. Fairly straightforward with PSExec \computername MSIEXEC /Uninstall {packageID} /qn.

    Problem is we have a number of versions of said software installed so the package ID could be different. I whipped together a quick VB Script that would loop through the remote registry looking for the "displayname" value and doing a simple string match. I tried it out on a test machine and it worked flawlessly.

    However, on one of the first "live" PCs I realized that many registry keys don't utilize the "displayname" value, which is especially worrisome if they happen to follow the one(s) that match.

    Seeing a bunch of psexec processes spin off when you know that there should only be one or two (at most) and knowing that each is running an MSIEXEC /uninstall? Makes you get real good at hitting Ctrl-C real fast.

    CAPTCHA - pointer (tried NULL, didn't work)

  • Carnildo (cs) in reply to B
    B:
    Aaron:
    One thing I don't get is... if it did a del *.* on the root directory... wouldn't that have JUST deleted the root directory?

    If you were running MS-DOS, that would have taken command.com and config.sys as POWs. No booting after that.

    No, you can boot just fine without config.sys, and booting without command.com will just give a "Interpreter not found" error. The killer is that it deletes MSDOS.SYS (ie. the DOS kernel).

  • shambo (cs) in reply to poopdeville

    Reminds me when I took out half the company's intranet. VS2005 has a handy dandy publish tool. You point it at a directory, it deletes everything and copies up your web app.

    What sucks is if you accidentally pointed it at wwwroot.

    The other WTF that day is when I learned the network group keeps the daily back-ups offsite. So even though this happened at 8 in the morning, the disks were already offsite.

  • B (unregistered) in reply to Carnildo
    Carnildo:
    The killer is that it deletes MSDOS.SYS (ie. the DOS kernel).

    My bad. I knew that del . in C:\ took out something that would cause the system to be unbootable.

    You don't want me to start about the time when I wrote a bash shell script which deleted /vmunix on a Unix box...

  • anonymous (unregistered)
    In fact, it hadn't been long ago that AJ worked with another developer on these functions. The developer asked all the right questions; How do we want to separate the files? and When I'm clearing out one of these folders, should I just do a delete *.krq?

    Anticipating that future changes might result in different file extensions going in the temp folder, AJ told the developer to "just delete .."

    Jerry's kind of the scapegoat here, isn't he? AJ is the dumbass who told the developer to delete . ... I don't see how Jerry's coding style has anything to do with it. It sounds like the code was correct until AJ and the other developer got their hands on it.

    Maybe Jerry should not have tried to reproduce a fatal error, but truthfully, I probably would have done the same thing (i.e., made sure that the bug was really as bad as it seemed, although I might have tried it with a vmware image or stomething like that.)

  • SomeCoder (unregistered)

    Yeah, this is really A.J.'s fault. First off, yes, he did violate YAGNI because that piece is unlikely to change. Make the extension a constant and if it does change, decide then if it needs to be configurable (which is very unlikely since if it changes, the code would have to be recompiled anyway).

    Also, how is this a WTF? It was caught in testing and never deployed to a customer. I hardly think that's a What The Fuck OR a Worse Than Failure.

  • Uber (unregistered)

    The real WTF is that we still use C:

    Where are A: and B:

    Buckinghamshire... awh! Crap!

  • poopdeville (cs) in reply to SomeCoder
    SomeCoder:
    Yeah, this is really A.J.'s fault. First off, yes, he did violate YAGNI because that piece is unlikely to change.

    Is it? How could you possibly know that? I'm pretty sure A.J. has a better grasp of the domain he was working in than you do, unless you're the submitter. His concerns very well might have been legitimate.

  • Renan_S2 (cs)

    I think that, if you deleted NTLDR from the C:\ of a Windows NT/2K/XP/(Vista?) system, it would refuse to boot with an "NTLDR missing" error.

    But normally NTLDR is hidden/read-only/system so, unless something is really messed up, it should not get deleted.

  • Sjoerd Verweij (unregistered)
    Comment held for moderation.
  • charlie (unregistered)

    The real WTF is that their filesystem doesn't have a permission system.

    Captcha: burned. Really burned.

  • SomeCoder (unregistered) in reply to poopdeville
    poopdeville:
    SomeCoder:
    Yeah, this is really A.J.'s fault. First off, yes, he did violate YAGNI because that piece is unlikely to change.

    Is it? How could you possibly know that? I'm pretty sure A.J. has a better grasp of the domain he was working in than you do, unless you're the submitter. His concerns very well might have been legitimate.

    How do you know that it is likely to change? You don't and neither do I. I just think it's probably unlikely to change, given the detail of the post.

    Even if it changes hourly, he should have known better than to use ..

    As someone else said, delete it via file handles or - if it really does change as often as you think it might - make it configurable.

    Again, it's not like it matters much because this was caught in testing. Had this change wiped out some client machines... I doubt A.J. would have submitted it.

  • Zygo (unregistered) in reply to boyd
    boyd:
    And btw the bug is not in the function deleting *.* but in the function that return c:\ as the temp directory.

    Actually the bug is that deleting . in c:\ did anything other than raise errors (restrictions on the directory are too weak or the privileges of the invoking user are too high), or remove data files owned by the user (which would have been bad, but not debilitating to the system) or the application (which were intended to be deleted anyway).

    The problem can be even worse on non-Windows systems. I've lost count of the number of times I've seen a maintainer script that looks like this:

    eval cat "$configFile" rm -rf "$dataDir"/*

    which is great if $configFile exists and contains a line that looks like 'dataDir=/foo'; however, if $configFile does not exist, or is empty, or doesn't bother to define dataDir, then the expanded command becomes

    rm -rf /*

    This too would be acceptable, provided it's executing as the owner of the software (instead of root)...which it invariably isn't...

  • Hum? (unregistered)

    SYS C:

  • Grant D. Noir (unregistered) in reply to Zygo
    Zygo:
    boyd:
    And btw the bug is not in the function deleting *.* but in the function that return c:\ as the temp directory.

    Actually the bug is that deleting . in c:\ did anything other than raise errors (restrictions on the directory are too weak or the privileges of the invoking user are too high), or remove data files owned by the user (which would have been bad, but not debilitating to the system) or the application (which were intended to be deleted anyway).

    The problem can be even worse on non-Windows systems. I've lost count of the number of times I've seen a maintainer script that looks like this:

    eval cat "$configFile" rm -rf "$dataDir"/*

    which is great if $configFile exists and contains a line that looks like 'dataDir=/foo'; however, if $configFile does not exist, or is empty, or doesn't bother to define dataDir, then the expanded command becomes

    rm -rf /*

    This too would be acceptable, provided it's executing as the owner of the software (instead of root)...which it invariably isn't...

    Thats why I'll always put an echo in front of frisky commands, before running them for real.

    This reminds me of a former cow-orker, who wrote a script to run as nightly cron-job. Of course, the last thing the script did was to take out its trash. All the trash.

    The next morning the box was .. acting up. Apparently something had wiped the system32 directory during the night.

  • real_aardvark (cs) in reply to Fnord
    Fnord:
    Do some developers have a problem with the concept of a "temporary" file? If the file is temporary, you should delete it as soon as you have finished with it. Not leave it on disk. If you leave it on disk, for whatever reason, it isn't temporary is it?

    This is obviously a Windows application so it would seem that using the application data folder would have been a better solution. Or actually tracking the handles of each "temporary" file that was created and deleting it via. the handle instead of the shotgun approach used here. Even if the wildcard was correct, it's still a dumb method.

    Ah yes, the old "Resource Acquisition is Initialisation" pattern. Which doesn't actually feature in any "Pattern" book of which I am aware. And in fact is a pretty piss-poor acronym, because we should all replace "Initialisation" with "Ownership." It's worth a baker's dozen of the other ones, though, and it's surprisingly rare in practice.

    Ignoring language wars, and just mentioning in passing that C++ has always been set up to do this sort of thing, it is indeed correct to say that, if you write to a temporary file, you should be responsible for deleting the bloody thing afterwards.

    However, the number of times I've seen Shell scripts, Perl scripts, and probably even Python and Ruby scripts, try to change directory relative to the root (Windows or Unix, who cares?), fail (thus leaving the poor little script at the root), and then "clean up" via rm . is truly humbling.

    Even idiots wouldn't do this on an interactive command-line.

    Why do idiots think that their program will be any less idiotic than they are?

    Is there some magic involved here that I don't see?

  • real_aardvark (cs) in reply to JJ
    JJ:
    I had the exact same experience at my company. Our uninstaller used to wipe out the entire computer. it wasn't to long after that happened that we started basking in the joy of VMs.
    Well, you can't really complain about that. You invoke the uninstaller. It uninstalls.

    Really, some people will just whinge about anything.

  • real_aardvark (cs) in reply to SomeCoder
    SomeCoder:
    Yeah, this is really A.J.'s fault. First off, yes, he did violate YAGNI because that piece is unlikely to change. Make the extension a constant and if it does change, decide then if it needs to be configurable (which is very unlikely since if it changes, the code would have to be recompiled anyway).

    Also, how is this a WTF? It was caught in testing and never deployed to a customer. I hardly think that's a What The Fuck OR a Worse Than Failure.

    Holy Mother of Perl, three in a row.

    This has nothing to do with YAGNI. Nothing; nothing at all.

    YAGNI, in the manner of most acronyms (I like using Perl, but I detest the cutesy acronymic apologies for actually structuring the languate properly in the first place), is quite simply an attempt to avoid explanation. Who sez I ain't gonna need it?

    The incantation "YAGNI" is usually deployed by mentally aberrant cult followers who truly believe that Design Is Never Necessary. Or Will Emerge. Presumably like Venus, from the sea, on half a clam shell. Bad news, if you're in Bangalore (which is now called Bengaluru, pop-pickers).

    Basically, nobody in their right minds would hard-code this as ".". A million programmers would invoke their own deity, call her Yagni, queen of destruction and occasional manifestation of Kali, and hard-code it as "*.crud." And they're still wrong.

    An actual programmer, as opposed to somebody who makes a living out of "programming," would either use RAII (see above), or, in extremis, write a cleanup routine, preferably virtual, that can make use of the name/suffix/whatever that was used to create the damn thing in the first place.

    May I be the first to volunteer to nail the ears of the original culprit to the nearest wall?

  • tin (cs)

    I call Urban Legend or similar...

    This was clearly DOS (or DOS with Windows <= 3.11). Windows >= 95 will throw all kinds of errors if you tried to delete everything. It would be fairly obvious what happened because of the errors.

    Anyway, in DOS, you can quite happily delete everything off C:... Or even format. But it still doesn't lock up. At worst, it will complain about COMMAND.COM being missing, and ask for it. You can point it to a copy on a floppy and it'll continue working. But... The vital 3 files for DOS booting are all hidden and system. DOS doesn't like deleting them because of this.

    So if this WTF is true, they not only recursively deleted all files, but they also coded it to delete HIDDEN and SYSTEM files too. That's highly unlikely given they decided deleting . was a good idea... That level of thinking doesn't usually consider file attributes.

  • lvb (unregistered)

    I had a similar issue. I had created 2 installers - one for the server and one for the client. The client installation ran from the server installation directory (after the server app was installed). I had missed something (can't remember now exactly) in the setup for the client. When a customer uninstalled a client it uninstalled the server installation (including the data files from the server). Luckily they had a backup. They were the first to have the new installation media so at least I didn't have to go around let our other clients know.

  • net-send (unregistered)

    Bungie had something like this in the Myth uninstaller, I think. They used to have a history page that mentioned it.

Leave a comment on “Good Thing we Tested It”

Log In or post as a guest

Replying to comment #:

« Return to Article