• Miles Thompson (unregistered) in reply to vt_mruhlin
    vt_mruhlin:
    Tim B:
    I haven't felt this bad for a robot since I saw A.I.!

    A.I. only made me feel bad for Stephen Spielberg. WTF was that crap at the end about being able to clone people's memories, but they die after a day. That's the stupidest thing I've ever heard.

    / Surprised how many people thought those things at the end were aliens, not advanced robots though. I have lots of stupid friends, I guess.

    those were robots?! Oh! No it makes (*a little bit more) sense.

  • JohnFx (unregistered) in reply to operagost

    All that is missing is a scripted WAV file to play "Wake Up Damnit!" every time it sprung into action. However, if the first server ever died completely it might be a pretty sad scene to see the second server performing CPR on the first and begging it to Wake up.

  • Joe in Australia (unregistered)

    That story is really, really sad.

  • Anonymouse (unregistered)

    Awwwww little it robot didn't even get a hug and the end of the story. Now I feel sad.

  • Beeblebrox (unregistered) in reply to arosebyanyothername
    arosebyanyothername:
    I find that Nagios for system monitoring and network managed PDUs are a much more effective and scalable solution.

    I too find that having a naggy Cisco IOS (Nagios) monitor the network for Distributed Interactive Simulation (DIS) Protocol Data Units (PDUs) is very scalable.

  • Necrophagist (unregistered)

    the definition of 'Hacking'.

  • Dave G. (unregistered) in reply to Zygo
    Zygo:
    I used to build a circuit with a 555 timer, a 74LS00 and a couple of 74LS191's which would count very slowly to 16 (it took about 4 minutes) but would reset the counter to zero every time the hard disk light blinked. The "overflow" output pin on the 4-bit counters was connected to the RESET signal on the motherboard. No disk activity for 4 minutes and the machine gets reset. With an appropriate interface chip the serial port could be monitored too (I used to need this when I ran a BBS).

    Isn't it possible with 555 timer, a resistor, a capacitor and just one 7400 series counter? I made a CAT5 cable tester (a very crude one) using just these components.

  • tamponrobot (unregistered) in reply to Josh

    that was gold - IT Tampon Robot - ha ha ha

  • John (unregistered)

    this is quite possibly the best WTF ever. The submitter should get a medal, or some kind of prize...

  • Robin Barooah (unregistered)

    Erm, funny (and horrific) as this story is, this is a common problem with a cheap commodity solution:

    http://www.google.com/search?hl=en&client=safari&rls=en-us&q=watchdog+timer+pci+card&btnG=Search

  • I call bull on the story's ending (unregistered) in reply to programmer x

    "the faulty server was taken offline and replaced with a new one working under a new IP address. During the swap, ITAPPMONROBOT was moved to a neglected corner of the server room, plugged back in, and promptly forgotten. It spent the last weeks of its life dutifully opening and closing its CD ROM drive every two minutes, reaching in vain for the restart button that it'd never touch again."

    So the new server still had the same problem? The CD eject was by the ping script, remember?

  • Chris (unregistered)

    The real WTF is they had to write a script to ping the host to check when it was up, rather than doing something sensible and use a tool like Nagios to monitor that, and more.

  • iMalc (unregistered)

    I don't care whether it's much of a WTF or not. It's just downright hillarious! Funniest solution ever!

  • huang's mom (unregistered)

    poor ITAPPMONROBOT.....

  • Lizbyrd (unregistered)
    You see IIS has a big flaw, it leaks memory badly, and it is exasperated when the ASP code uses V.B.

    I agree, VB is pretty exasperating... C# all the way! ducks and runs

  • (cs) in reply to Jimboooo!
    Jimboooo!:
    Did that last bit make anyone else feel a bit sad?

    Yes, I just want to give the poor 'puter a loving home for the remains of it's life.

  • MichaelC (unregistered) in reply to I call bull on the story's ending
    I call bull on the story's ending:
    "the faulty server was taken offline and replaced with a new one working under a new IP address. During the swap, ITAPPMONROBOT was moved to a neglected corner of the server room, plugged back in, and promptly forgotten. It spent the last weeks of its life dutifully opening and closing its CD ROM drive every two minutes, reaching in vain for the restart button that it'd never touch again."

    So the new server still had the same problem? The CD eject was by the ping script, remember?

    Ummm, no it didn't have the same problem. As it says, the new server was commissioned with a new IP address, so pinging the old server's IP address would result in no response.....and trigger the eject sequence :)

  • (cs)

    The real WTF? is, that none of you super computer guys know anything about RESETATOR piece of hardware. It's simple small piece of HW, which sits on serial port waiting for a ping. If no come in some period, then it triggers switch, which may be connected to machine reset button. There are also enhanced GSM versions, which can reset computer over GSM network :)

    Oh, you really need to learn... :))

  • D2oris (unregistered) in reply to cybershaman
    cybershaman:
    The real WTF? is, that none of you super computer guys know anything about RESETATOR piece of hardware. It's simple small piece of HW, which sits on serial port waiting for a ping. If no come in some period, then it triggers switch, which may be connected to machine reset button. There are also enhanced GSM versions, which can reset computer over GSM network :)

    Oh, you really need to learn... :))

    Who cares, this story is much better. Poor little robot. I want to squeeze and hug it.

  • frogking (unregistered) in reply to Josh

    IT Tampon Robot, no .. Strapon Robot, yes ..

  • Arturo Ronchi (unregistered)
    ...It spent the last weeks of its life dutifully opening and closing its CD ROM drive every two minutes, reaching in vain for the restart button that it'd never touch again.
    Thats pure poetry !
  • Anonymouse (unregistered)

    That's so sad :(

  • defproc (unregistered)

    Ha. Nice. The wording of the last paragraph reminded me, for some reason, of Kubrick's "AI".

  • (cs) in reply to Me
    Me:
    Actually, I did the same thing back in 2001ish using one Linux server to keep another up. The positioning was the hardest thing.

    Redundant Array of Inexpensive Servers?

    I wonder if you could expand that - a circle of four servers, each one resets another one if it goes down. I guess you'd have to do some mechanical work to ensure they can press each other's reset button.

    (Yes, I know that primitive circuitry would allow a computer to power-cycle another one. But where's the Rube Goldberg fun of that?)

  • Cloak (unregistered) in reply to yet another Matt
    yet another Matt:
    I loved that story. I feel sorry for the poor machine in the corner tirelessly doing it's job. I want to give it a nice home.

    How come it still opens and closes? Is the server still not OK? At least this poor little machine is still doing its job.

  • Cloak (unregistered) in reply to yet another Matt
    yet another Matt:
    I loved that story. I feel sorry for the poor machine in the corner tirelessly doing it's job. I want to give it a nice home.

    How come it still opens and closes? Is the server still not OK? At least this poor little machine is still doing its job.

  • LeadFoot (unregistered) in reply to Josh

    You have dylsexia too?

  • Zygo (unregistered) in reply to asdf
    asdf:
    wait, so it'd potentially reset a perfectly working machine simply because it hadn't seen disk activity in 4 minutes?

    Technically since the device was looking for edges in the LED signal, it would also reset the machine if a single access took more than 4 minutes (because for example your disk drive firmware has crashed)...but yes.

    Modern server machines have watchdog devices like this built into the chipset or buried in the IPMI subsystem. Once you enable the device, the OS has to send a signal to it every N seconds for some configurable value of N, or the machine gets reset. Usually you have some program running on the machine that tests your machine's favorite service and then tells the OS to zero the watchdog timer. If your favorite service hangs, so does the test program, then the watchdog timer expires and the machine resets. Ditto if the OS crashes.

    My timer boards were in use a decade or so before anything "modern," and I used an existing signal from the motherboard to avoid having to build interface circuitry. Using the hard disk LED pin or RS-232 TXD pin was easy since there were standardized connectors for these signals. No OS-specific device drivers required either, just write something to the disks or send something through the serial port every few seconds--things healthy systems typically do already.

    When the machines I fitted with these timers are running their normal workloads one can safely assume that if more than 15 seconds pass between disk accesses the machine has crashed, or is in a sufficiently degraded state to be equivalent to crashed for all practical purposes.

    The 4 minute timeout was only as long as it was to accomodate the BIOS power-on routines. Most of the time the counter was held at "0" during normal operation (it takes 15 seconds to reach "1"). Later versions of the board used a LED to display the clock signal just to make the board look like it was doing something during normal operation.

    There was also a bypass switch on the output of the timers so you could spend more than 4 minutes fiddling with the BIOS or running diagnostic software or something.

  • Simon J (unregistered)

    The last paragraph nearly brought a tear to my eye...

  • Zygo (unregistered) in reply to Dave G.
    Dave G.:
    Isn't it possible with 555 timer, a resistor, a capacitor and just one 7400 series counter? I made a CAT5 cable tester (a very crude one) using just these components.

    Almost. The 555 gets unstable if you make its period too long. At the extremes the clock period is unpredictable or it stops entirely. I found that I could get to about 15 seconds without much jitter, which means 16 clock pulses take 4 minutes. If it wasn't for this problem I wouldn't need the digital counter at all: I could just configure the 555 timer to cycle once every 4 minutes and reset it with the HDD LED input.

    I'm pretty sure you need some kind of logic chip too--at least one inverter, because most interesting signals from the counter chip (overflow, MSB, etc) are active-high and RESET is active-low.

    I needed to detect transitions in the HDD LED signal because the OS might crash during a disk access. If I detected only the LED-on state then such a crash would hold the counter in its reset state and the machine would not be rebooted.

    When I designed the board I had a lot of 4-bit counters left over from a cancelled project. I used two counters on the board, one which counted when the LED was on and one when the LED was off. The two counter overflow outputs were ORed together and the (inverted) result went to the ~RESET line on the PC.

    If I used only one counter I'd need some other chip to detect edges in the HDD LED signal. Since I had lots of counter chips for free, the economics of the alternatives didn't work out.

    Some prototype versions of the board had lots of unnecessary parts from the spares'n'leftovers bin: 7-segment LED decoders (driving 7-segment LED displays of mismatched sizes and colors), or demultiplexers driving bar-graph LED arrays, sometimes even a third counter that just counts up to drive blinkenlights.

  • (cs) in reply to I call bull on the story's ending
    I call bull on the story's ending:
    So the new server still had the same problem? The CD eject was by the ping script, remember?

    I call bull on the rumor that you have a brain.

    The new server had no problem. The CD eject was by ping script, but the ITAPPMONROBOT machine pinging the NEW IP address assigned to the server when it was moved into the corner.

    You seriously couldn't figure that out? Even though the article specifically mentions the server's new IP address? I hope you don't work for me.

  • Zygo (unregistered) in reply to Richard Sargent
    Richard Sargent:
    I am surprised no one commented on the WTF of an undocumented server. If Erik had documented its purpose, there is a chance someone might have read the documentation when they went to move the monitoring server and realized it was no longer needed. Note: I said "a chance"! :-)

    Remember that the server has "DO NOT MOVE" written on it. Did you think that someone who didn't read the text printed on the machine would read documentation from some other place?

  • Zygo (unregistered)

    A few people here could use a technology update...

    These days even sub-$100 UPSes have USB monitoring and control. At the very least you can command the UPS to power off. If AC power is present the UPS will power on again some seconds later (good UPSes will allow you to configure this, not-so-good ones do this after a fixed delay); otherwise the UPS stays off until power is restored (good UPSes will also wait until a minimum battery charge is present, not-so-good ones will power on anyway and might fail if AC power dies again soon after). This is a pretty basic capability that is required for controlled system shutdown and automatic restart--even the proprietary 3-wire RS-232 port interfaces provide a signalling mechanism for this.

    Modern Unix systems (even Linux) have had journalled filesystems for years now. Database servers and commercial UNIX filesystems have implemented their own journalling or logging for years before that. Even NTFS is reasonably bulletproof against worse failure modes than a system lockup (although the shoddy quality of typical software for systems that actually use NTFS tends to negate this advantage). These systems are designed to cope with unexpected system halts and resets, but not necessarily power failures. In the ITAPPMONROBOT case it seems we are dealing with a non-power-failure case, since the machine has stopped responding to pings for up to two minutes before it is reset.

    In cases where a server is strictly locking up (i.e., the system crashes before writing any corrupt data to disk, the disks are fundamentally healthy, and power is maintained to the disks throughout) there should be no data loss, or well-defined data loss (e.g., the last N transactions committed might not be replayed during startup recovery, depending on configuration). Many crashes fit within those criteria.

    New high-performance system designs take into account the fact that system power fails slowly and not all at once--your RAM can lose power before your disk drives and disk controller, causing any data writes that were initiated before the power failure to store junk on the disk. This can do some serious damage to your data even if you are using all the standard software journalling/logging capabilities.

  • Zygo (unregistered)

    I think I found ITAPPMONROBOT's new gig:

    http://questionablecontent.net/view.php?comic=5

  • an actual coder (unregistered) in reply to Josh
    Josh:
    Did anyone else read the title as IT Tampon Robot? I thought this story was going to take a HUGE twist.

    Damn dyslexia.

    I read it as "I tap mon robot", and silently translating "mon" into "my" from French.. More than dyslexia going on with me, I fear.

  • (cs) in reply to cybershaman
    cybershaman:
    The real WTF? is, that none of you super computer guys know anything about RESETATOR piece of hardware. It's simple small piece of HW, which sits on serial port waiting for a ping. If no come in some period, then it triggers switch, which may be connected to machine reset button. There are also enhanced GSM versions, which can reset computer over GSM network :)

    Oh, you really need to learn... :))

    Since you will have noted the fact that there was no money budgeted to purchase hardware, and you wouldn't have mentioned it unless they could have gotten one anyway, where do they give these resetators away?

  • Nande! (unregistered)

    The best history i ever read in TDWTF :D

  • Random Passerby (unregistered) in reply to Domo Arigato
    Domo Arigato:
    Wouldn't a UPS controlled by the 2nd machine have beeen easier? Just cycle it when the machine stopped responding.

    cough "budget freeze" cough

    I'm reminded of one of my favourite quotes, which Erik seems to exemplify:

    “We are the willing, led by the unknowing, Doing the impossible, for the ungrateful. We have spent so long doing so much with so little, We are now qualified to do anything with nothing.”

  • (cs) in reply to ITAPPMONROBOT
    ITAPPMONROBOT:
    I agree this was brilliant. UPC costs money and unless the BIOS is a relatively new one, the server will still require manual intervention after the UPC kicks in.
    I think you mean UPS, and you are incorrect because a really old server will have hard power (thus the circuit will remain complete as long as the power button is down) and a somewhat newer server will ALWAYS have power on settings in the BIOS. I haven't seen an ATX-based server without "full on" settings in its BIOS... ever.
  • Beentheredidnotdothat (unregistered) in reply to Domo Arigato

    Not necessarily. We have a remote Dell Server, that when powered off at the wall, requires a finger to power up, rendering the IP managed power strip it sits on, useless. Bad Dell!

  • Old School Code Slinger (unregistered) in reply to programmer x

    That is great! Add it to the manual! I can see it now:

    When server stops working:

    1. Position body sagittal to side of server box.
    2. Raise foot off floor by bending knee. Either foot is appropriate.
    3. Aiming toes toward chalk circle on side of server box, expand muscles of leg so as to kick the chalk circle.
    4. Be sure to make contact. Apply enough force so that server box is not knocked over.
    5. If server does not respond and continue operating hit the reset button.
  • token whinger (unregistered)

    tldr looked crap

  • (cs)

    This is easily the greatest Daily WTF of all time. It has the lowly staff triumphing in the face of management stupidity! It has the greatest hack known to man. It has a sad story of one brave soldier abandoned after he has done his duty.

    I will now cycle my CD drive to salute this steadfast little machine.

    Seriously the only way this would be better if it was a precisely aligned USB Foam Dart Missile Launcher ;)

  • Matt Elmore (unregistered)

    Why not just get an APC Masterswitch?

    Those have been around for a very long time. Allow you to power on/off/cycle outlets remotely via a web interface or telnet...

  • John (unregistered)

    It reminds me of the quote that goes something like:

    'If it is a stupid idea, and it works, then it is not a stupid idea.'

  • (cs) in reply to Cloak

    It is a really sad story, deep down each of us realizes that we are that robot, reaching for something we once had.

    Or we just anthropomorphize the little robot way too much.

  • (cs) in reply to programmer x

    MacGyver!!

  • Messenger61 (unregistered)

    Given the right hardware setup you could even make two servers monitor each other (and reboot each other)...

  • JimM (unregistered)

    The real wtf is that so few commenters read the actual article before commenting! Phrases like "budget freeze", "turn of the 21st century" and "worked for years" are passed over so they can demonstrate their knowledge of modern technology that should've been used to provide a more elegant solution. Apart from this is mid-90s technology in a BUDGET FREEZE (even inexpensive new hardware would've been infrastructure change and would need management clearance - chance!). And so many people have either missed "new IP address", OR don't understand ping (although surely the old server's IP address would've been recycled and assigned to another machine?).

    Of course, the really elegant solution would've been to strip down the CD drive and fashion a small lightweight extending arm (probably out of a pencil or a biro tube) which would've been easier to position; or could've driven a seesaw type rocker from on top of the faulty server. But that's just being hyper critical, and this is a lovely story...

  • (cs) in reply to Zygo
    Zygo:
    A few people here could use a technology update...

    At least one person here could use a common sense/reading comprehension/sense of humor update. Let me see if I can help that person out. (That person would be you, in case you have trouble figuring it out.)

    1. No budget to replace existing hardware in the story.

    2. None of what you rambled on and on about has any bearing.

    3. Nothing in the story gave a date (even approximately) indicating when this happened, so it could have been prior to any of the stuff you ranted about being available.

    4. Probably 99.9999% of the people here knew what you were spouting off about, so you wasted our time telling us things we knew already in an attempt to make yourself appear smarter than you obviously are (otherwise, you wouldn't have made this senseless posting).

    5. It annoys people when you post something in the tone you used (that of a pompous know-it-all).

Leave a comment on “ITAPPMONROBOT”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article