• (cs)

    What's the WTF? We've been trained to disbelieve all but the core of the submitted story, so the only thing that I believe from this story is that there was a motherboard with a broken clock, but that's really scraping the barrel.

  • Mike (unregistered) in reply to pjt33
    pjt33:
    What's the WTF? We've been trained to disbelieve all but the core of the submitted story, so the only thing that I believe from this story is that there was a motherboard with a broken clock, but that's really scraping the barrel.

    I assume having identified a fault bad enough to take the server away from one service, they just put the bad server on a different (previously working) service.

    Oh, and (temporially) FRIST

  • QJo (unregistered) in reply to pjt33
    pjt33:
    What's the WTF? We've been trained to disbelieve all but the core of the submitted story, so the only thing that I believe from this story is that there was a motherboard with a broken clock, but that's really scraping the barrel.

    No, it's an infrastructure configuration bodge which was diagnosed (incorrectly) as being a broken motherboard clock.

    As in: when you can't apply the appropriate problem-solving techniques to debugging your substandard software, blame it on the hardware.

  • QJo (unregistered) in reply to QJo
    QJo:
    pjt33:
    What's the WTF? We've been trained to disbelieve all but the core of the submitted story, so the only thing that I believe from this story is that there was a motherboard with a broken clock, but that's really scraping the barrel.

    No, it's an infrastructure configuration bodge which was diagnosed (incorrectly) as being a broken motherboard clock.

    As in: when you can't apply the appropriate problem-solving techniques to debugging your substandard software, blame it on the hardware.

    Sorry, ignore me -- I misread (and, ahem, misunderstood) the story.

  • Torque (unregistered)

    Expected some retarded script that changed the time a bit whenever it ran. Was let down. F-, would not read again.

  • (cs)

    I've never known computers to query the hardware real-time clock except at power on. The kernel clock is generally much more accurate that the RTC.

    Wherever the problem is, it isn't the motherboard's RTC.

  • Greg (unregistered)

    The temporal lab people actually made a buggy time machine that keeps traveling between those two times?

  • ANON (unregistered)

    Strange open ending. Reminds me of the president's sick daughter.

  • (cs) in reply to Torque
    Torque:
    Expected some retarded script that changed the time a bit whenever it ran. Was let down. F-, would not read again.
    I read comments like this, and I keep hearing this instead: "Instead of office chair, package contained bobcat, would not buy again"...
  • Craig S (unregistered) in reply to Steve The Cynic

    If I got a bobcat instead of a chair, I would definately buy it again.

  • Fritz, a.k.a. Fritzo (unregistered) in reply to Craig S

    I would definitely sit on a bobcat. One of the few things I've not sat on. I mean I've sat on Bob, and I've sat on a cat. This has the potential to combine the advantages of both.

  • anonymous (unregistered)

    Isn't the ending missing? If not, not mentioning the "cutting corners" bit would have given the whole story a more complete feel.

  • JohanE (unregistered) in reply to Craig S
    Craig S:
    If I got a bobcat instead of a chair, I would definately buy it again.

    http://www.nba.com/bobcats/2013-14-lady-cats :-)

  • faoileag (unregistered)

    From Linux System Administrator's Guide @ about.com: "date only shows or sets the software clock" (emphasis by me).

    A hardware problem? Unlikely. A kernel problem? Unlikely. Some homegrown script trying to keep several servers synced on the same system time? Unlikely, after all two successive "date" calls deliver two different timestamps and the pattern looks too regular for a race condition (date calls vs. script).

    An executable named "date" that only serves one of two hardcoded dates alternatively? Now that's the most plausible scenario so far.

    Anyone with a better idea?

  • Peter Wolff (unregistered) in reply to Greg
    Greg:
    The temporal lab people actually made a buggy time machine that keeps traveling between those two times?
    They forgot the inherent quantum nature of time traveling. What time you end at, can't be determined until you get there.
  • Ol' Bob (unregistered) in reply to Mike
    Mike:
    pjt33:
    What's the WTF? We've been trained to disbelieve all but the core of the submitted story, so the only thing that I believe from this story is that there was a motherboard with a broken clock, but that's really scraping the barrel.

    I assume having identified a fault bad enough to take the server away from one service, they just put the bad server on a different (previously working) service.

    Well, y'see, getting a new mo-board in would require a manager to sign off on the purchase, which would require said manager to admit that there was a problem and, more to the point, that there was a problem which had not been anticipated. Now, any good manager knows that they should budget for the problems they anticipate having - this is just good management. But there is, of course, no need to budget for unanticipated problems because you can't see them coming, so you can't plan for them, so you can't budget for them, so there really isn't a problem, now is there? Thus, the occurrence of an unanticipated problem is a direct statement that the manager did not anticipate this problem occurring and is, therefore, a poor manager. Now, even the poorest manager knows that the road to happiness and mega-bonus-bucks does NOT traverse the dreaded Desert of Truth. No, the road to happiness and etc is a sixteen-lane expressway that runs across the fecund Mud Plains of Creative Expressivity (also known as the Swamp of Bullsh*t), with a possible side excursion into the Back Alley of CoverUp. Now, imagine the quandary into which our manager is thrown by this occurrence. On one hand, there's a VPN server with a busted clock which keeps chucking a user off. OK, it's only one user - but you know how some of these people are - pretty soon she'll be mentioning it to her manager, who will mention it to his VP, who will talk to the VP over our particular manager, who will land on said manager like a ton of bricks. Thus, there's plenty of reasons to get the problem fixed on the quiet-and-cheap. So, what's a harried manager to do? Sign the PO for a new motherboard? NO! That would raise Questions. Words would be Spoken. Looks would be Looked. And before you know it our poor manager would be out on the streets, looking for a new career in the Fast Food Industry from whence he/she/it came. No! This cannot be! There must be Another Way - and lo, Another Way was found!

    So they just switched the VPN and FTP servers.

    CATCHA: haero - And in the end the manager was a big haero.

  • Ol' Bob (unregistered) in reply to Peter Wolff
    Peter Wolff:
    Greg:
    The temporal lab people actually made a buggy time machine that keeps traveling between those two times?
    They forgot the inherent quantum nature of time traveling. What time you end at, can't be determined until you get there.

    Are we there yet? And are we having fun yet?

    CAPTCHA: iusto - We'll be there in iusto minute.

  • faoileag (unregistered) in reply to anonymous
    anonymous:
    Isn't the ending missing? If not, not mentioning the "cutting corners" bit would have given the whole story a more complete feel.
    I read it that Karl's department trumped IT and requisitioned one of their servers (running as ftp server), so the IT department had to use the malfunctioning server as ftp server after that. And that IT either didn't order the new server or that that server, when it arrived, was requisitioned by Karl's department as well.

    CAPTCHA: suscipit - IT department suscipit nullam?

  • ¯\(°_o)/¯ I DUNNO LOL (unregistered)

    It's simple. They reused the server because they needed one that was bigger on the inside.

  • isaturnine (unregistered) in reply to robbak

    Also, most time synchronization software, certainly *nix ntp clients, will only ever set the clock backwards when very explicitly told to do so. On *nix monotonic time is considered almost axiomatic and something a programmer will assume (s)he can rely on. Do not confuse this with the way the clock is apparently changed backwards once a year due to DST - this only changes the result of system calls that you use when you WANT time already translated to your local time zone and DST rules, and will NEVER mess with what time(2) returns. Setting the actual clock backwards on a production *nix machine is something you [i]would preferrably want to reboot it after[i], there are more and worse things than VPN software that can crash or misbehave (for example, a classically implemented cron or atd will become confused and execute things [i]out of order[i]).

    The whole thing looks like someone inexpertly jury rigged a time synchronization mechanism from something like rdate and a loop...

  • (cs) in reply to ANON
    ANON:
    Strange open ending. Reminds me of the president's sick daughter.
    Rule number one for the president's sick daughter: do not mention the president's sick daughter.

    Rule number two for the president's sick daughter: write a bad article with no clear WTF and an ending that redefines open endings to move the discussion away from the president's sick daughter.

  • (cs)

    Frist?

    (Reference to the time travel stuff!)

  • (cs) in reply to Torque

    I was expecting the screen scraping scripts to mess about with the network interface/routes/whatever so they could connect to the target server (thus killing the VPN connection temporarily), then change it back again.

    As it is I don't see a wtf.

  • anon (unregistered) in reply to Cratig
    Cratig:
    Frist?

    (Reference to the time travel stuff!)

    Tanks for explaining that. It makes it a lot more funny.

  • faoileag (unregistered) in reply to TGV
    TGV:
    ANON:
    Strange open ending. Reminds me of the president's sick daughter.
    Rule number one for the president's sick daughter: do not mention the president's sick daughter.
    You are breaking rule number one regarding she-who-must-not-be-mentioned.
  • M-x org-mode (unregistered)

    "What's costing Chronos so much they can't even afford a new server that can keep time?"

    You know, for an used roadworthy DeLorean you have to pay at least 30.000$. Not to mention the plutonium costs. The last time the deal with the Lybians got awry, so we had to contact more expensive suppliers.

  • foo AKA fooo (unregistered)
    but every half hour or so it would disconnect.
    Alex glanced at her phone. "It'll happen in an hour. Let's give it a try."
    Temporally Confused, yeah.
  • foo AKA fooo (unregistered) in reply to Torque
    Torque:
    Expected some retarded script that changed the time a bit whenever it ran. Was let down. F-, would not read again.
    Agreed. Erik was on a good way, but this a relapse to last years's levels. s/Karl/Hanzo/;s/Alex/Gertrude/;s/Temporal Lab/Hesse University/;s/project/ninja/ and it all seems awfully familiar.
  • LordHighFixer (unregistered)

    The problem was simple really. Someone has a job setup to sync the clock every hour using two different time sources to be sure. However, they used ntpdate instead of setting up a proper ntpd configuration.

    Captcha: letatio - How to make a salad really happy.

  • MyNameIs (unregistered)

    I see Erik's not got the hang of writing complete stories yet.

  • poster of the screenshot (unregistered) in reply to faoileag
    faoileag:
    From Linux System Administrator's Guide @ about.com: "date only shows or sets the software clock" (emphasis by me).

    A hardware problem? Unlikely. A kernel problem? Unlikely. Some homegrown script trying to keep several servers synced on the same system time? Unlikely, after all two successive "date" calls deliver two different timestamps and the pattern looks too regular for a race condition (date calls vs. script).

    An executable named "date" that only serves one of two hardcoded dates alternatively? Now that's the most plausible scenario so far.

    Anyone with a better idea?

    If it was a script I never found it and it must have been installed at the dc.

  • qazwsx (unregistered) in reply to Torque
    Torque:
    Expected some retarded script that changed the time a bit whenever it ran. Was let down. F-, would not read again.

    You were expecting a problem caused by a brain-dead hack, and what do you know, it's an Erik Gern story.

  • Black Bart (unregistered) in reply to isaturnine
    isaturnine:
    The whole thing looks like someone inexpertly jury rigged a time synchronization mechanism from something like rdate and a loop...

    This! And the reason they happened on that solution is that they were running the server in a primitive virtualization environment which didn't properly implement the timekeeping resources that the OS uses for RTC. It was probably too unstable and too far off so that NTP refused to correct it. So they just did the periodic rdate trick with all the associated problems.

  • faoileag (unregistered) in reply to poster of the screenshot
    poster of the screenshot:
    faoileag:
    An executable named "date" that only serves one of two hardcoded dates alternatively?
    If it was a script I never found it and it must have been installed at the dc.
    I must admit that I suspected the supplier of the wtf of having fabricated it.

    Since you as the supplier take care to answer me, this theory can now be best described as being "obviously wrong".

    But since you are here: care to shed some light on how much of the story is your submission and how much is embellishment?

  • MrBester (unregistered)

    "We'll order a new chassis" Yeah, because when a motherboard has a problem the fix is always to replace the box it lives in...

  • Schmitter (unregistered)

    Surprised that they didn't just abandon VPN and or FTP support.

  • v (unregistered)

    Well, the screenshot is fake for sure, because on it the submitter has root access, and no sane person would grant him that. Or if someone did, they would at least blame them for all the problems that occur with that system afterwards.

  • vt_mruhlin (unregistered)

    The "frist" posters really aren't dedicated to their craft enough. Would it have killed somebody to make the same "frist" post twice, 1 minute and 7 seconds apart from each other?

  • D'Arque Bishop (unregistered)

    I'm surprised that with all of the time travel jokes, no one has noticed the subtle(?) Doctor Who reference: the machine name is "river" (as in River Song, the Doctor's wife), and the subject of the story is named Alex (as in Alex Kingston, the actress who plays River).

    (Oh God, I'm such a geek...)

  • (cs)

    Let me see if I can guess how this all got started.

    "The time on this server is off. Fix it!" "Um, okay... I usually right click on the clock for that but... um... here's a command I can use. I'll just paste that in and run it."

    # rdate server1

    "Huh. The time still keeps slipping. Maybe I should just put it in one of those loopy things I saw."

    # while 1 ; do rdate server1 ; done

    "There. Fixed."

    (Time passes.)

    "Y'know, I just read about a scheduled job thingy called cron. Maybe I should put that rdate-ily-doodle into there and run it every half hour."

    0,30 * * * rdate server2

    (More time passes. Sometimes forwards, sometimes backwards.)

    "Time's wrong. Do that scripty thing that you did to fix it."

    "Um, okay. Didn't I already do that? Oh well, guess it isn't working. Here goes..."

    # while 1 ; do rdate server3 ; done

    "Done. I'm sure that this will fix the problem and never create any other issues."

    (Despite the best efforts of three different time changing processes, time still passes)

    "Say... What's a 'Tier 1 NTP server'? Maybe I should get time from that too..."

  • anonymous (unregistered) in reply to faoileag
    faoileag:
    From Linux System Administrator's Guide @ about.com: "date only shows or sets the software clock" (emphasis by me).

    A hardware problem? Unlikely. A kernel problem? Unlikely. Some homegrown script trying to keep several servers synced on the same system time? Unlikely, after all two successive "date" calls deliver two different timestamps and the pattern looks too regular for a race condition (date calls vs. script).

    An executable named "date" that only serves one of two hardcoded dates alternatively? Now that's the most plausible scenario so far.

    Anyone with a better idea?

    I was halfway expecting some kind of round robin setup where somehow it's running every command on one of two systems whose clocks aren't synchronised.

  • Anomaly (unregistered)

    The WTF is taking a known to be malfunctioning server that causes disconnects and using for a service that requires connections be maintained. IE from FTP to VPN. So now instead of disconnecting when you try to download a large file, have remoting in for the day as you'll be disconnected every hour.

    Thats a WTF.

  • (cs) in reply to Fritz, a.k.a. Fritzo
    Fritz:
    I would definitely sit on a bobcat. One of the few things I've not sat on. I mean I've sat on Bob, and I've sat on a cat. This has the potential to combine the advantages of both.
    You've sat on Bob? Are you the retarded son he writes about so much?
  • (cs) in reply to faoileag
    faoileag:
    poster of the screenshot:
    faoileag:
    An executable named "date" that only serves one of two hardcoded dates alternatively?
    If it was a script I never found it and it must have been installed at the dc.
    I must admit that I suspected the supplier of the wtf of having fabricated it.

    Since you as the supplier take care to answer me, this theory can now be best described as being "obviously wrong".

    But since you are here: care to shed some light on how much of the story is your submission and how much is embellishment?

    Well, his username says what his submission was.

    Everything else is embellishment!

  • Ian (unregistered)

    I suspect virtualisation, some versions of Xen run time backwards occasionally.

    https://wiki.debian.org/Xen/Clocksource

  • (cs)

    ?this.Comment.IsFrist false ?this.Comment.IsFrist true ?this.Comment.IsFrist false ?this.Comment.IsFrist true ?this.Comment.IsFrist false ?this.Comment.IsFrist true ?this.Comment.IsFrist false ?this.Comment.IsFrist true

  • (cs) in reply to faoileag
    faoileag:
    From Linux System Administrator's Guide @ about.com: "date only shows or sets the software clock" (emphasis by me).

    A hardware problem? Unlikely. A kernel problem? Unlikely. Some homegrown script trying to keep several servers synced on the same system time? Unlikely, after all two successive "date" calls deliver two different timestamps and the pattern looks too regular for a race condition (date calls vs. script).

    An executable named "date" that only serves one of two hardcoded dates alternatively? Now that's the most plausible scenario so far.

    Anyone with a better idea?

    Issue at Erik's company that he's trying to have us solve? Probably.

  • Marvin the Martian (unregistered) in reply to D'Arque Bishop
    D'Arque Bishop:
    with all of the time travel jokes, no one has noticed
    No one has pointed out the source of the 'bobcat' reference either. No one has pointed out what this 'email-thingy' is doing not in a compliant email address, nor why this little wave follows it. Some things are assumed.

    I accidentally wrote #*$% as source for the feline, but then went in my time machine to correct it; that's also how I finally finished my highschool with good grades next year.

  • n_slash_a (unregistered) in reply to faoileag
    faoileag:
    From Linux System Administrator's Guide @ about.com: "date only shows or sets the software clock" (emphasis by me).

    A hardware problem? Unlikely. A kernel problem? Unlikely. Some homegrown script trying to keep several servers synced on the same system time? Unlikely, after all two successive "date" calls deliver two different timestamps and the pattern looks too regular for a race condition (date calls vs. script).

    An executable named "date" that only serves one of two hardcoded dates alternatively? Now that's the most plausible scenario so far.

    Anyone with a better idea?

    It seemed that rather than order a new server, they just switched the VPN and FTP servers. So instead of the VPN randomly crashing, a FTP transfer will occasionally fail. Probably better, since most FTP clients can handle dropped chunks, but still a WTF that the root cause wasn't fixed.

  • Anomaly (unregistered) in reply to n_slash_a
    n_slash_a:
    faoileag:
    From Linux System Administrator's Guide @ about.com: "date only shows or sets the software clock" (emphasis by me).

    A hardware problem? Unlikely. A kernel problem? Unlikely. Some homegrown script trying to keep several servers synced on the same system time? Unlikely, after all two successive "date" calls deliver two different timestamps and the pattern looks too regular for a race condition (date calls vs. script).

    An executable named "date" that only serves one of two hardcoded dates alternatively? Now that's the most plausible scenario so far.

    Anyone with a better idea?

    It seemed that rather than order a new server, they just switched the VPN and FTP servers. So instead of the VPN randomly crashing, a FTP transfer will occasionally fail. Probably better, since most FTP clients can handle dropped chunks, but still a WTF that the root cause wasn't fixed.

    It was the other way around. They replaced a perfectly good VPN server with a broken server that disconnects all the time.

Leave a comment on “Temporally Confused”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article