• root (unregistered) in reply to eric76
    eric76:
    I wonder why they didn't just set the following in /etc/ssh/sshd_config?

    AllowUsers = root craig carl

    At the very least, they should also have:

    PermitRootLogin without-password PasswordAuthentication no

    And much better would be

    PermitRootLogin no PasswordAuthentication no AllowUsers = craig carl eric76

    Of course, the AllowUsers would also include everyone else who might need to use ssh to access the host.

    No need for the equals sign. You can just do 

    AllowUsers craig carl eric76

  • Mr. AHole DBA (unregistered) in reply to lol
    lol:
    not malice? Yet they had a script that deliberately kicked other users off the system.

    I would have wiped the entire server and rebuilt it - like a virus infestation, I'd never consider it safe unless it was totally disinfected.

    And I'd also install aide to keep an eye on what else they might do in the future... with a view to getting them sacked with extreme prejudice!

    PS. yes, I enjoyed this one too.

    In all fairness, they were kicking people off what they thought was their development server, but also allowed root access at least.

    If it's your dev server, do what you want, that's one reason why root exists.

  • (cs) in reply to root
    root:
    eric76:
    I wonder why they didn't just set the following in /etc/ssh/sshd_config?

    AllowUsers = root craig carl

    At the very least, they should also have:

    PermitRootLogin without-password PasswordAuthentication no

    And much better would be

    PermitRootLogin no PasswordAuthentication no AllowUsers = craig carl eric76

    Of course, the AllowUsers would also include everyone else who might need to use ssh to access the host.

    No need for the equals sign. You can just do 

    AllowUsers craig carl eric76

    I just checked and see that on our machines, some have the '=' and some do not. I never noticed that before.

    I wonder if sshd matches '=' as a permitted user or just ignores the '='.

  • (cs) in reply to Jay
    Jay:
    There's also the query of either why Russell was deploying to "Craig and Carl's devlepment server", or why Craig and Carl had attempted to appropriate an in-use server for apparently personal purposes.
    This ambiguity is explained in the HTML comments:
    HTML:
    He laughed a grim laugh and emailed a copy of the script to their manager. It was bad enough they'd used the dev team's server without asking permission and broken its provisioning, trying to lock everyone else out should be the last straw.
    So Craig and Carl act like being still in kindergarten - believing they could claim a server by mobbing all other users.
  • Steve Wahl (unregistered) in reply to faoileag
    faoileag:
    And another victim of the world wide web: using the underline as means to emphasize one or more words!

    Actually, as I understand it, you're supposed to use italics for emphasis when available, and use underlining for emphasis when italics are not available (say you're using a typewriter or a daisy wheel printer). (In the old days, Nroff put underlines where Troff put italics…)

    So it actually may be reasonable that the web re-purposed the underline.

  • (cs) in reply to cellocgw
    cellocgw:
    Hey! My name is Carl, you insensitive clod!

    (note to self: disallow root logins on the server I "hacked")

    I thought that you weren't going to tell anyone about us!

  • garaden (unregistered) in reply to no laughing matter
    no laughing matter:
    This ambiguity is explained in the HTML comments:
    HTML:
    He laughed a grim laugh and emailed a copy of the script to their manager. It was bad enough they'd used the dev team's server without asking permission and broken its provisioning, trying to lock everyone else out should be the last straw.

    What what what? Thanks for the tip, had no idea those were there!

    Also, this is wonderful. Obviously, the WTF is logging onto a misbehaving server only to see THIS IS CRAIG AND CARL'S DEVLEPMENT SERVER right in the MOTD. I can imagine the sounds of furious typing from Russell's cube suddenly stopping, then a soft "clunk" when his jaw hits the desk. Amazing.

  • (cs) in reply to Zylon
    Zylon:
    The two had been roommates in college and took a strange pride in telling Russell how they'd been too busy "hacking" (their word) to pass most of their classes.

    Really Dan? You put the word "hacking" in quotes, to signify that it was a quote, then felt the need to further inform us that someone was being quoted? Really?

    Ironically, the quotes aren't there to indicate a quote.

  • Captain Obvious (unregistered)

    When standards and process start getting implemented, tin pot fiefdoms like these are some of the first to be swept up.

  • ¯\(°_o)/¯ I DUNNO LOL (unregistered) in reply to Zylon
    Zylon:
    Really Dan? You put the word "hacking" in quotes, to signify that it was a quote, then felt the need to further inform us that someone was being quoted? Really?
    "hacking" was in "scare quotes"
  • (cs) in reply to camelotbob
    camelotbob:
    Does the name Puppet not make anybody else think of a bad low budget horror movie?

    I just think of a quote from Ghost in the Shell: "the Puppetmaster? He's nothing but a puppet himself!"

  • Hermun (unregistered)

    So why did Russell think he owned Craig & Carl's server? I can request something to be done to a server but if it's not mine I shouldn't throw a tantrum if it's not done.

  • (cs) in reply to garaden
    garaden:
    What what what? Thanks for the tip, had no idea those were there!

    Yes, much like the alt text on xkcd, there is usually more to stories on here hidden in HTML comments. I usually only bother checking them for Remy though, as he uses them more often than some of the others.

  • (cs) in reply to Hermun
    Hermun:
    THIS IS CRAIG'S DEVLEPMENT COMMENT

    get out of my comment!

  • Paul Neumann (unregistered) in reply to ¯\(°_o)/¯ I DUNNO LOL
    ¯\(°_o)/¯ I DUNNO LOL:
    "hacking" was in "scare quotes"
    Educational
  • LarryLaffer (unregistered)

    TRWTF was that Russel was using Puppet. Ansible is the far better solution.

  • Ren Bussell (unregistered)

    It must be some coincidence that I wore a free t-shirt I got from Puppet Labs today.

  • Bill C. (unregistered) in reply to chubertdev
    chubertdev:
    cellocgw:
    Hey! My name is Carl, you insensitive clod!

    (note to self: disallow root logins on the server I "hacked")

    I thought that you weren't going to tell anyone about us!
    That's what I thought too. See what you get from thinking too much?

  • Bill C. (unregistered) in reply to Nagesh
    Nagesh:
    Matt Westwood:
    "scrubber" in the UK is a word which was popular in the 1970s and 1980s to mean what you Americans usually call a "ho". AKA: "slapper".

    In India scrubber is something we use to scour vessels. There are two kind of scrubber that are more commonly available. Plastic fibers! [image] Steel Wool! [image]

    Now American company called 3M is also selling some product that does job of scrubber. It is expensive compare to other product. [image]

    It doesn't matter to me if the scrubber is using those products or not. If the scrubber is as cute as the pickle slicer, I'm in.
  • Norman Diamond (unregistered) in reply to tin
    tin:
    Mark:
    Has Windows implemented^Wimitated this yet?
    Sudo for Windows? I'm not sure if this is the one I've tried or not... I'm not using anything like this on Windows at the moment.
    Oh no. Now Windows is going to make me a sandwich? But I don't want to be a sandwich!
  • Norman Diamond (unregistered) in reply to Steve Wahl
    Steve Wahl:
    faoileag:
    And another victim of the world wide web: using the underline as means to emphasize one or more words!
    Actually, as I understand it, you're supposed to use italics for emphasis when available, and use underlining for emphasis when italics are not available (say you're using a typewriter or a daisy wheel printer). (In the old days, Nroff put underlines where Troff put italics…)

    So it actually may be reasonable that the web re-purposed the underline.

    You bet. Now when you put links in your typewritten material, you have to put them in italics.

  • Russell (unregistered)

    I'm the guy who submitted the story.

    Many liberties were taken with it (I didn't actually set up the puppet infrastructure, I maintained it after an architect did so), but the core is correct. They took over the server so that they could have a server to play with - they had messed up the apache configuration and the routine puppet deployment failed because of that.

    The names were made up. I'm saying that because Remy came unwittingly rather close with one of them. :P

    I did immediately gather evidence and I reloaded the server. No way was I going to try to salvage that mess.

  • Russell (unregistered)
    Also, this is wonderful. Obviously, the WTF is logging onto a misbehaving server only to see THIS IS CRAIG AND CARL'S DEVLEPMENT SERVER right in the MOTD. I can imagine the sounds of furious typing from Russell's cube suddenly stopping, then a soft "clunk" when his jaw hits the desk. Amazing.

    Yes, pretty much. See, the thing is - I liked the guys. I didn't have anything against them personally. But that really took balls. I mean, really? Just waltz in and take over a server? What motivates someone to do something so obviously wrong?

    They were not fired, but I do know for a fact that their upward mobility in the company was affected. One was angling for a sysadmin job. That was pretty much immediately taken off the table.

    I still have nothing against them, but they got off easy. Professionally... I would have at least written them up for it. You just don't do that.

    Meh. This was years ago. Funny how it's one of the few things that stick in my head. One of the guys went on to troll the company gift exchange... but that's a story for another time.

  • coyo (unregistered)

    Would you rather be the Puppet Master or the Muppet Pastor?

    captcha: cogo. go co go

  • anon (unregistered) in reply to Frank
    Frank:
    Seriously, why do some sites go to all the bother of creating links and then style them into oblivion? Or, similarly, obfuscate them behind a bunch of script nonsense so that they act like links (sometimes) but don't parse like links?

    <button class="btn btn-link">Brought to you by Twitter Bootstrap</button>

    It's a thing. http://getbootstrap.com/css/#buttons

  • (cs) in reply to camelotbob
    camelotbob:
    Does the name Puppet not make anybody else think of a bad low budget horror movie?
    Mm, ham! I love it!

    But here's a suggestion: if you're setting up a payoff for the end of the story, don't put it in the title.

  • (cs) in reply to Klimax
    Klimax:
    (And don't forget Xenix)
    Damn you. I've been trying to forget all about Xenix for over 2 decades, and now you tell me I shouldn't?
  • (cs) in reply to Russell
    Russell:
    One of the guys went on to troll the company gift exchange... but that's a story for another time.

    I for one would like to hear another Craig and Carl story.

  • Klimax (unregistered) in reply to nerd4sale
    nerd4sale:
    Klimax:
    (And don't forget Xenix)
    Damn you. I've been trying to forget all about Xenix for over 2 decades, and now you tell me I shouldn't?

    Long before my time. But I would suspect there are far better candidates for loss of memory...

    distineo is another destination.

  • (cs) in reply to cmccormick
    cmccormick:
    Russell:
    One of the guys went on to troll the company gift exchange... but that's a story for another time.

    I for one would like to hear another Craig and Carl story.

    +1

    much better than Hanzo

  • 1234 (unregistered) in reply to TRWTF

    I know, huh!

  • Russell (unregistered)
    I for one would like to hear another Craig and Carl story.

    Let's just say some poor woman got a 24 pack of toilet paper at the Christmas gift exchange and leave it at that. I felt so bad about it I bought her a gift card to Coffee Bean.

  • Russell (unregistered)

    This was actually quite an interesting workplace.

    • We had some pool noodles and figured out that a dry erase marker was just the right size to turn it into a gigantic oversized blow dart. All fun and games until the dry-erase blowdart smacked into a manager's window while they were having a meeting. As far as I know, there's still a red dot on the wall where the cap flew off from the impact.

    • One coworker used to love to light hand sanitizer on his file cabinet. I am so surprised he never got caught.

    • We did up that coworker's cube while he was on holiday - absolutely everything got wrapped. I have pictures somewhere.

    • That coworker found a stress ball, and tore it apart. Inside that stress ball, he found a ball of corn starch. So, hearing it was non-newtonian, he put it on the floor, found a sledgehammer, and just started WHALING on it. 20 strikes later, a maintenance person came by with the radio blaring "yeah, someone's pounding on the floor up there, can you look into it?" Cure "deer in the headlights" look, dropping of the hammer, and running off into the next room.

    • And, we actually had a pretty good test one day of "he who smelt it, dealt it", leading to the entire netops/nops team sitting in the reception area refusing to go back until the smell dissipated.

    ... it was fun in one way, but one of the most unprofessional places I've ever worked, and I'm glad that era's over.

  • (cs) in reply to eric76
    eric76:
    I wonder why they didn't just set the following in /etc/ssh/sshd_config?

    Probably because one of the first things anyone who deploys a configuration management tool to production does is have it manage sshd_config, and their changes would have been over-written the next time Puppet ran?

  • anonymous (unregistered) in reply to The Great Lobachevsky
    The Great Lobachevsky:
    garaden:
    What what what? Thanks for the tip, had no idea those were there!

    Yes, much like the alt text on xkcd, there is usually more to stories on here hidden in HTML comments. I usually only bother checking them for Remy though, as he uses them more often than some of the others.

    I got tired of viewing the source and wrote a Greasemonkey script that shows comments (and for good measure, any elements in the comments that have CSS display:none).

  • kingbeardo (unregistered)

    Granted it was misspelled, but could Russell not figure out that 'devlepment' meant development? You don't mess with someone's development server without asking! Maybe Craig and Carl had a good reason to have a non-standard setup that only allowed the two of them and root to have open ssh sessions (though the scrubber 'solution' is a bit stupid) -- the only way to know would be to ask, which Russell the up-jumped IT drone didn't bother to do. If anyone should be fired it should be Russell.

  • (cs) in reply to kingbeardo
    kingbeardo:
    Granted it was misspelled, but could Russell not figure out that 'devlepment' meant development? You don't mess with someone's development server without asking! Maybe Craig and Carl had a good reason to have a non-standard setup that only allowed the two of them and root to have open ssh sessions (though the scrubber 'solution' is a bit stupid) -- the only way to know would be to ask, which Russell the up-jumped IT drone didn't bother to do. If anyone should be fired it should be Russell.

    I agree with one part of what you said. I don't think you understand who the guilty party was, though.

  • Russell (unregistered) in reply to chubertdev
    chubertdev:
    I agree with one part of what you said. I don't think you understand who the guilty party was, though.

    This "up-jumped IT drone" isn't going to feed the troll. :)

  • anonymous (unregistered) in reply to kingbeardo
    kingbeardo:
    Granted it was misspelled, but could Russell not figure out that 'devlepment' meant development? You don't mess with someone's development server without asking! Maybe Craig and Carl had a good reason to have a non-standard setup that only allowed the two of them and root to have open ssh sessions (though the scrubber 'solution' is a bit stupid) -- the only way to know would be to ask, which Russell the up-jumped IT drone didn't bother to do. If anyone should be fired it should be Russell.
    I realise this may be too subtle of a difference for a troll to detect, but Craig and Carl were part of NOC - network operations - and the servers belonged to DevOps - developer operations. Craig and Carl were not developers; they were trained monkeys whose job was to keep the blinkenlights flashing so that the actual developers could do developer-type work.
  • Neil (unregistered) in reply to A Guy
    A Guy:
    faoileag:
    Taco:
    Warren:
    Anyone try clicking "done" in paragraph 3, simply because it was underlined, even though there was no logical reason for such a word to link to anything and the links are blue on TDWTF as illustrated in paragraph 2? No? Me neither....
    I did, I regret it now, I wonder if I will recover fully....
    And another victim of the world wide web: using the underline as means to emphasize one or more words!
    Could be worse
    That's so cute...
  • solar (unregistered)

    TRWTF is allowing devops (glorified term for cs janitors) to muck about with dev machines in the first place.

  • Russell (unregistered) in reply to solar
    solar:
    TRWTF is allowing devops (glorified term for cs janitors) to muck about with dev machines in the first place.

    Technically we were not devops. We were systems engineers. We deployed system software using puppet as well as applications, and IIRC the developers controlled their own puppet manifests through SVN (if they didn't, that should have been the case). We had root on everything and did system deployments (VM, etc) as well.

    I have only once worked at a place where I didn't have root, and I ended up working around that pretty well.

    Put another way, you're far too opinionated about things you don't understand.

  • (cs) in reply to Russell
    Russell:
    chubertdev:
    I agree with one part of what you said. I don't think you understand who the guilty party was, though.

    This "up-jumped IT drone" isn't going to feed the troll. :)

    I just find it really ironic that he's so right about one part, and then completely mishandles that advantage.

  • eric bloedow (unregistered)

    "Our procedures are so unsophisticated that trained monkeys could run the NOC." that reminds me of story: a supermarket used electronic cash registers, which frequently crashed. BUT, for some reason i can't recall (union rules?), the employees were not permitted to fix the registers themselves, they had to call the support company, which the author of that story called "Howler monkeys Inc." to come over and reset the master register..and they were ABSURDLY incompetent, often taking HOURS to fix the problem, partly because they couldn't tell which register was the master, DESPITE the employees telling them "This one! THIS ONE!" i don't remember how that story ended...

Leave a comment on “Fear the Puppetmaster”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article