• (cs)

    Fristly, time zone of that server should be set correctly. See man 5 crontab

  • EuroGuy (unregistered)

    It would have been more interesting to know what the reason for that cron job was...

  • (cs)

    So did they add a signal handler to log that an external signal requested the application to shut down?

  • Stephen (unregistered)

    A simple, but apparently effective, form of "chaos monkey".

  • (cs)

    Never attribute to malice that which is adequately explained by stupidity.

  • Doctor_of_ (unregistered)

    What could possibly have come to the mind of the writer of this amazing piece of chaos? Was he bullied by Steve and his co-workers and this was his revenge? Did a program with PID 21342 once shared his pics on internet? Or is he The Dude stuck in the The Grid and 21342 was the PID of Clu? Infinite possibilities. All delicious.

  • Lauren (unregistered)

    If the application stays up all the time until the cron kills it, why would its PID suddenly match the one specified on the crontab? tl;dr - PIDS don't change.

  • (cs)

    I don't think I would have caught that. Good one on Steve!

  • John (unregistered) in reply to Doctor_of_

    I can see how it could arise.

    Someone is testing the signal handling of a process. It's currently 12:15 and he wants to see what happens if it gets an automated signal from a cron job. His process is already running with a pid of 21342, and so he edits a quick entry into crontab - but then forgets to remove it afterwards.

    Since there were apparently lots of entries there, all essential to the running of the system, when the code was promoted to live, that entry came along too.

  • Dulrich Repper (unregistered) in reply to Lauren
    Lauren:
    If the application stays up all the time until the cron kills it, why would its PID suddenly match the one specified on the crontab? tl;dr - PIDS don't change.
    By "stays up" maybe he means that it respawns itself periodically.
  • (cs) in reply to Lauren
    Lauren:
    If the application stays up all the time until the cron kills it, why would its PID suddenly match the one specified on the crontab? tl;dr - PIDS don't change.

    Because the author neglected to mention this (other) cron entry:

    12 21 * * * /etc/init.d/uber-application restart
  • (cs) in reply to John

    I can see how you never heard of the sleep command.

  • GWO (unregistered)

    Chaos Monkey cronscript:

    [ $RANDOM -lt 100 ] && killall mysqld

  • Cronboy (unregistered)
    12 22 * * * kill 21342 Every day, at 12:22PM, the system ...

    You meant "22 12 * * * kill 21342", right ? Otherwise this was a really late lunch !

  • (cs) in reply to snoofle
    snoofle:
    So did they add a signal handler to log that an external signal requested the application to shut down?
    Nobody can handle SIGKILL, except for Doug McIlroy.
  • (cs) in reply to Cronboy

    Man, now I have to check the email. I'm 99% certain I copied it straight out of the submitter's message that way, but it's still possible I rekeyed it and jacked things up.

  • JayGee (unregistered)

    Should have set it to run at 11:22, sleep for an hour, then use the OMGWTF 2 code to randomly determine if it kills the process today or not. Beautiful sabotage.

  • Prefect (unregistered)

    That would have worked once in every 274 years. And that's assuming the application gets restarted (i.e. its PID changes) every day. If it stays up for days/weeks, it may take time up to the next ice age until it gets really killed by that.

  • Chris (unregistered) in reply to shinobu

    But kill normally sends SIGTERM, which can be handled.

  • -is (unregistered) in reply to GWO
    GWO:
    Chaos Monkey cronscript:

    [ $RANDOM -lt 100 ] && killall mysqld

    Linux weenie. As opposed to Solaris weenie, who would by the same method shutdown the system to single user mode.

  • (cs) in reply to Prefect

    The original submission didn't explain the mechanisms, but I'm going to assume that we're dealing with something that spawns lots of child processes, and doesn't handle it gracefully when one of the children dies. Just a guess, I don't know.

    Also, the actual problem-solving process took over a year before anybody really got annoyed. Presumably, it wasn't happening all that frequently.

  • Stephen (unregistered) in reply to Cronboy
    Cronboy:
    12 22 * * * kill 21342 Every day, at 12:22PM, the system ...

    You meant "22 12 * * * kill 21342", right ? Otherwise this was a really late lunch !

    The other possibility being the company operates in multiple timezones with the core being somewhere else.

  • Charles F. (unregistered)

    I know what happened.

    A business manager said, "It's only a matter of time before this application crashes mysteriously," and a development manager wrote it down as a requirement.

  • EvilSnack (unregistered) in reply to Doctor_of_

    This is the good-old-fashioned sort of WTF for which I come to this site to read. It's not TDPHB or the Daily Dogbert or the Daily Elbonian, but something typed into computer file that causes trouble and makes your head explode trying to think of a good reason for it to have been typed.

    Captcha: abbas. "The first ABBA was the best. Now we have a bunch of second-rate abbas."

  • (cs) in reply to shinobu
    shinobu:
    snoofle:
    So did they add a signal handler to log that an external signal requested the application to shut down?
    Nobody can handle SIGKILL, except for Doug McIlroy.
    And Chuck Norris.
  • QJo (unregistered)

    The real WTF is Turkish pizza, amirite?

  • Fishcake (unregistered)

    The WTF is that Steve's Borek was still hot. He'd left the cafe, got back to the office, fixed the issue and it was still piping-hot!

  • (cs) in reply to QJo

    Turkish pizza is not worse than failure- it's better than success. My arteries are hardening just thinking about it.

  • (cs) in reply to QJo
    QJo:
    The real WTF is Turkish pizza, amirite?

    I'd say TRWTF is ignorant new worlders calling it turkish pizza, because they don't know the word Lahmacun, but they have heard of turkey and pizza.

  • (cs) in reply to nobulate
    nobulate:
    Never attribute to malice that which is adequately explained by stupidity.
    This is a good guideline most of the time. But taking this attitude can also aid the malicious.
  • (cs) in reply to eViLegion

    Lahmacun is not the same thing as Turkish pizza. At least in the Turkish places around here, Turkish pizza is someplace between lahmacun and pide. Admittedly, a subtle distinction, but it's nothing like the traditional lahmacun.

    It's sort of like getting Chinese food in the US. It's nothing like actual Chinese food. While Turkish pizza is closer to actual Turkish food, it's been deeply Americanized.

  • Toasty (unregistered)

    TRWFT is that he thinks a sandwich is healthy.

  • (cs)

    TRWTF is the screenshot of Mac OS X: a platform where cron has been deprecated in favour of launchd.

  • someguy (unregistered) in reply to Doctor_of_
    Doctor_of_:
    What could possibly have come to the mind of the writer of this amazing piece of chaos? Was he bullied by Steve and his co-workers and this was his revenge? Did a program with PID 21342 once shared his pics on internet? Or is he The Dude stuck in the The Grid and 21342 was the PID of Clu? Infinite possibilities. All delicious.

    Sounds exactly like somebody ran

    `echo "12 22 * * * kill $(pidof thing_that_needs_to_be_reset)" >> /etc/crontab"'

    or somesuch - i.e. when entered the command was something sane, but it got evaluated a bit sooner than expected. Still a WTF on the part of the author of that command, I'd say.

    (Of course, this would be a classic 'disgruntled ex-employee' move, too)

  • (cs) in reply to nobulate
  • (cs) in reply to nobulate
    nobulate:
    Never attribute to malice that which is adequately explained by stupidity.
    Until you realize it's both.
  • Black Bart (unregistered)

    The real WTF is that now I'm hungry for Turkish Pizza and Borek but I dont' even know what they are.

  • (cs) in reply to Remy Porter
    Remy Porter:
    Lahmacun is not the same thing as Turkish pizza. At least in the Turkish places around here, Turkish pizza is someplace between lahmacun and pide. Admittedly, a subtle distinction, but it's nothing like the traditional lahmacun.

    It's sort of like getting Chinese food in the US. It's nothing like actual Chinese food. While Turkish pizza is closer to actual Turkish food, it's been deeply Americanized.

    Yeah, but calling it turkish pizza is like calling a curry "Indian Stew", because it's been Anglicized. Anyone who does that needs pain.

  • (cs) in reply to Mr. DOS
    Mr. DOS:
    TRWTF is the screenshot of Mac OS X: a platform where cron has been deprecated in favour of launchd.

    ...as demonstrated by the zero-length /etc/crontab :)

  • Dan Brown (unregistered)
    Between piping-hot bites of borek
    The atrocious writing style and editing has really put me off reading TDWTF.
  • (cs) in reply to Dan Brown
    Dan Brown:
    Between piping-hot bites of borek
    The atrocious writing style and editing has really put me off reading TDWTF.

    At least we didn't have to pay for this, nor experience every TV channel in the world telling us it's the best story ever.

  • (cs) in reply to Mr. DOS

    Read the source, Mr. DOS.

  • Geoff (unregistered) in reply to John
    John:
    I can see how it could arise.

    Someone is testing the signal handling of a process. It's currently 12:15 and he wants to see what happens if it gets an automated signal from a cron job. His process is already running with a pid of 21342, and so he edits a quick entry into crontab - but then forgets to remove it afterwards.

    Since there were apparently lots of entries there, all essential to the running of the system, when the code was promoted to live, that entry came along too.

    Using cron for that is a WTF, there is tool for running something just once at a specific time, ( at ).

  • Daniel (unregistered)

    Small math correction to something that was noted above: on a somewhat regular system, this would have a 1/32768 chance of killing a process with randomly varying PID each day, and thus only have a 50% probability of happening once in

       50% = [complement of not happening]^[this many times in a row]    ½ = (1−1/2¹⁵)^(365.25⋅{years})    ⇒ ln(½)/ln(1−1/2¹⁵)/365.25 = 62 years.

    Either $PIDMAX was set to something much lower (but still higher than 21342, so that is not really plausible), or the process owned a multitude of PIDs that all caused the application to crash on SIGTERM. To crash with 50% certainty every 6 months, it would have to average on 124 vulnerable PIDs at any given (lunch)time.

  • (cs) in reply to QJo
    QJo:
    The real WTF is Turkish pizza, amirite?

    Maybe author want us to goggle for the recipe.

  • Neil (unregistered)

    Who uses 82×24?

  • (cs) in reply to shinobu
    shinobu:
    snoofle:
    So did they add a signal handler to log that an external signal requested the application to shut down?
    Nobody can handle SIGKILL, except for Doug McIlroy.
    But even Doug can't handle the consequences of "kill -9 1".
  • (cs) in reply to Neil
    Neil:
    Who uses 82×24?
    Remy Porter?
  • Uli (unregistered)

    The name is Börek, you can't spell turkish food without umlauts.

  • ZoomST (unregistered) in reply to Daniel
    Daniel:
    ... or the process owned a multitude of PIDs that all caused the application to crash on SIGTERM. To crash with 50% certainty every 6 months, it would have to average on 124 vulnerable PIDs at any given (lunch)time.
    Very believable since most core application behemoths are badly grown apps that will spawn and re-spawn processes as if the server possesses bazillions of processors and memory. And having 1 process killed without notice (not any process) could make the app to go "save everything and close" mode, aka Steve's boss panic.

Leave a comment on “A Crony Joke”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article