• John (unregistered)

    And - most places I've worked at have done this, at least to some extent.

    Financial institutions really ought to know better though

  • Dlareg (unregistered)

    I'm so happy I'm the only user of my programs :D So I'm only annoying my self when shit happens.

  • (cs)

    Why does this remind me of a German U-Boat commander at the tail end of the Second World War?

  • (cs)
    Any objections raised by the developers were met with the same reaction you’d get from a dog after explaining the Pythagorean Theorem – blank, drooling faces.

    You have a pretty stupid dog

  • (cs) in reply to taylonr
    taylonr:
    Any objections raised by the developers were met with the same reaction you’d get from a dog after explaining the Pythagorean Theorem – blank, drooling faces.

    You have a pretty stupid dog

    Yeah, my dog does logarithms now.

    I'm having problems getting it to learn English though, so he can't tell me his answers.

  • Waffle (unregistered)

    Goddamn stupid captcha. I want my comment posted now! get posted ! GET POSTED !!!!

  • Archibald Buttocks (unregistered)

    And this is why I stopped developing trading software - it's a fast-track to an early stress-induced death, and the pay is nothing short of dire. And the software is always crap because there's a constant push to have it all done yesterday, sod quality.

  • (cs)

    For some reason, I'm reminded of some of the quotes from the turrets in Portal...

    Deploying!

    Critical Error!

    Shutting down!

  • (cs) in reply to Archibald Buttocks
    Archibald Buttocks:
    And this is why I stopped developing trading software - it's a fast-track to an early stress-induced death, and the pay is nothing short of dire.

    Is the pay really that bad? I'd imagine they'd pay top dollar for bearing such horrible stress. A mistake at our business (real-time service, but each transaction is small) can cause money to bleed, but nothing like the stock market.

  • JC (unregistered) in reply to John
    John:
    And - most places I've worked at have done this, at least to some extent.

    Financial institutions really ought to know better though

    Financial institutions seem to be the worst for this sort of stuff in my experience. I used to work at Lehman Brothers and we had several similar instances caused by lack of testing and a general, we need it in there ASAP attitude.

  • (cs) in reply to NaN
    NaN:
    ... Yeah, my dog does logarithms now.

    ...

    Logarithms by NaN's dog. Something is wrong here...

  • (cs) in reply to Kefer
    Kefer:
    NaN:
    ... Yeah, my dog does logarithms now.

    ...

    Logarithms by NaN's dog. Something is wrong here...

    It could be worse... it could be Schrödinger's cat!

  • Mike (unregistered) in reply to Kefer
    Kefer:
    NaN:
    ... Yeah, my dog does logarithms now.

    ...

    Logarithms by NaN's dog. Something is wrong here...

    They're all NaN. No wonder his dog can do them...

  • Kaos (unregistered)

    History repeats itself…

    The Nasdaq OMX in Sweden has had problems both yesterday (40 minutes) and today (5.5 hours), after an update of the SAXESS software. According to the news the problem is that no one can login.

    Carl Norell, responsible for media contacts at Nasdaq OMX in Sweden:

    "We understand that this has caused problems for our customers. But despite everything, many have been sympathetic. They know that something like this has not happened for a long time.”

    (This is the longest stop since March 1999 when it was closed for a whole day. Reason? Deployment of SAXESS)

  • (cs) in reply to JC
    JC:
    John:
    And - most places I've worked at have done this, at least to some extent.

    Financial institutions really ought to know better though

    Financial institutions seem to be the worst for this sort of stuff in my experience. I used to work at Lehman Brothers and we had several similar instances caused by lack of testing and a general, we need it in there ASAP attitude.

    This mirrors my (limited) Wall St. experience, too. Everyone in management hated the idea of testing. Most clients just could not grasp the concept of testing. Yes, you're buying extra hardware for the QA layer but you will save thousands (millions?) in the long run, dammit! Clients that actually did buy servers for QA would often neglect them so that they became wildly out of sync with their production systems.

    Eventually, the company motto became, "We'll test it in production!"

  • (cs)

    I've worked for Wall Street brokerages for nearly 20 years now, and it's ALWAYS code/cursory test/deploy. Screw QA and their time-wasting procedures. Screw the deployment team (we don't care about keeping DR versions of software up to date, we're working in PROD).

    However, I've found that the pay, if you're a US citizen, has been quite good - more than double what I'd make in a comparable non Wall Street position.

    One thing I've learned; if your boss is one of these we-don't-need-to-test jerks, just blindly obey and let it hit the fan. The next time around, they usually soften their stance a bit. If not, let it hit the fan even bigger. As long as you CYA with memos detailing the risk, you are blameless, and come across after-the-fact as having tried to remedy the problem. In the short run you lose; in the long run you win.

  • (cs)

    No need for preveiwing! Is the comment on the site yet? What are you waiting for?! Submit! SUBMIT!

  • (cs)
    the amount of lost commissions, sales, and trades outweighed the cost of testing and fixing the change over ten-fold
    Management thinking: yes, but as long as you hit problems less than one time in ten, you're up.
  • AdT (unregistered)

    Wait... they had a separate development environment? That's not following proper developmestuction procedure. Deployment in real developmestuction is when you hit the Save button in your source code editor!

  • (cs)

    "De ploy! De ploy!" "My dear guests, I am Mr. Roarke, your host. Welcome to Fantasy Testing."

  • James M (unregistered) in reply to NaN
    NaN:
    Yeah, my dog does logarithms now.

    My dog just did a really big logarithm, right on the carpet in front of me

  • Biff (unregistered)

    So that's what happened to Martin Prince...

    </obscure simpsons reference>

  • Aijaz (unregistered) in reply to FredSaw

    Best. Comment. Ever. (especially since it was only yesterday that I was trying to explain the concept of Fantasy Island to a coworker)

  • Ron Larson (unregistered)

    I'm a long-time (25+ years) systems developer. I've seen this attitude too. However, in this case, I would blame the IT manager, not management. He failed for the following reasons.

    (a) No recovery plan (b) No audit plan

    Any software, no matter how well tested, might fail. So there must be a recovery plan. Also, because software might fail, there must be a method in place to audit it to assure that it functioned properly.

    The recovery plans and procedures, and the audit systems, must be be in place first, before any development of any production software is in place.

    IT will always be under extreme pressure. That is part of the job. And yes, no testing is a mistake. But not planning for failure was a bigger mistake.

  • Nicholas (unregistered)

    You know, the real problem in this scenario is with the lousy developers. They should just frickin' get it right the first time. I don't know how many hundreds of people I've met who really just can't be trusted to code. Use your brains, people! Don't call it done if it isn't done! Think about all the code paths! Learn the system you're working with. And don't whine about not having the time. Just remember this mantra, which has always served me well: no matter how much you think it's somebody else's fault, it's really your fault. And your problem. You have no idea the world of problems this attitude will fix, and how far it will take you. Though it may just make you frustrated with everybody else's incompetence.

  • (cs) in reply to Outlaw Programmer
    Outlaw Programmer:
    Eventually, the company motto became, "We'll test it in production!"
    One place I worked, the new project manager handed out pin-on buttons. They were just plain white, with the letters "JSI" in black, and the universal circle-and-slash over them in red.

    She forbid the developers from continuing their traditional practice of, if the application compiled, to just ship it to the customers.

  • WallStreetDeveloper (unregistered) in reply to QuinnFazigu

    I was surprised he said the pay was dire. Where else can you earn $200k+ as a developer?

  • Rob (unregistered) in reply to Nicholas

    For the love of God, I seriously hope you're being sarcastic.

  • Biff (unregistered) in reply to Nicholas
    Use your brains, people! Don't call it done if it isn't done! Think about *all* the code paths! Learn the system you're working with.
    Well, I imagine that works if you're writing 50 line Basic utilites. Unfortunately in the real world life isn't so simple.

    Presumably you're also one of those people who doesn't believe testing is necessary as your code is invariably flawless...

  • (cs) in reply to Nicholas
    Nicholas:
    no matter how much you think it's somebody else's fault, it's really your fault.
    Mom? Is that you?
  • (cs) in reply to snoofle
    snoofle:
    I've worked for Wall Street brokerages for nearly 20 years now, and it's ALWAYS code/cursory test/deploy. Screw QA and their time-wasting procedures. Screw the deployment team (we don't care about keeping DR versions of software up to date, we're working in PROD).

    However, I've found that the pay, if you're a US citizen, has been quite good - more than double what I'd make in a comparable non Wall Street position.

    One thing I've learned; if your boss is one of these we-don't-need-to-test jerks, just blindly obey and let it hit the fan. The next time around, they usually soften their stance a bit. If not, let it hit the fan even bigger. As long as you CYA with memos detailing the risk, you are blameless, and come across after-the-fact as having tried to remedy the problem. In the short run you lose; in the long run you win.

    Believe me, if you've obeyed your nincompoop boss (who ironically originated in QA) and installed something that brings, say, the entire CompuServe network down, thus requiring a manual restart on every single end-node, this is not a working philosophy that you're ever going to carry forward into a future job.

    Next time I'll just hit the bastard over the head with a large hammer. No jury of my peers would convict me.

  • Frank (unregistered)

    Back in the day, I worked for a derivatives trading desk in Chicago, and this rings so true.

    Don't blame the IT manager; there isn't one ... a business guy is calling all the shots. Don't fault the testers; there aren't any. The developers usually came straight out of college, or from a similar firm; even if they've taken the time to research best practices, they have their business-side boss wanting everything yesterday. And I was in the comparatively sedate development team; the programmers who actually sit with the traders literally have to grind out new programs the moment a trader cracks his whip.

    I once had a boss who, when I suggested some modest improvements to our process, replied "We're a trading firm, not a software company." Which relies entirely on in-house software ...

  • loki (unregistered) in reply to Biff
    Biff:
    Use your brains, people! Don't call it done if it isn't done! Think about *all* the code paths! Learn the system you're working with.
    Well, I imagine that works if you're writing 50 line Basic utilites. Unfortunately in the real world life isn't so simple.

    Presumably you're also one of those people who doesn't believe testing is necessary as your code is invariably flawless...

    Hey! He may have a lot of faults, but being wrong isn't one of them! Just ask him...

  • (cs) in reply to AdT
    AdT:
    Wait... they had a separate development environment? That's not following proper developmestuction procedure.
    For the love of god, just let that awful neologism die.
  • rumpelstiltskin (unregistered) in reply to NaN
    NaN:
    taylonr:
    Any objections raised by the developers were met with the same reaction you’d get from a dog after explaining the Pythagorean Theorem – blank, drooling faces.

    You have a pretty stupid dog

    Yeah, my dog does logarithms now.

    I'm having problems getting it to learn English though, so he can't tell me his answers.

    You can't teach a dog to "tell" you the answers, because his speech system is too primitive. You have to teach him to write the answers down.

  • (cs) in reply to Frank
    Frank:
    Don't fault the testers; there aren't any... the programmers who actually sit with the traders literally have to grind out new programs the moment a trader cracks his whip.
    I can endure a lot of worst practices at $200/hr.
    1. Do what the boss says.
    2. ???
    3. Profit!
  • matt (unregistered) in reply to NaN
    NaN:
    taylonr:
    Any objections raised by the developers were met with the same reaction you’d get from a dog after explaining the Pythagorean Theorem – blank, drooling faces.

    You have a pretty stupid dog

    Yeah, my dog does logarithms now.

    I'm having problems getting it to learn English though, so he can't tell me his answers.

    My dog is awesome.

    I ask my dog what the derivative of y = (5ln(4x) + y root(10) - 5 * 10010^(5x)) * (9 - (33)) is, and he says nothing.

    Beat that.

  • Hans (unregistered) in reply to Kaos
    Kaos:
    The Nasdaq OMX in Sweden has had problems both yesterday (40 minutes) and today (5.5 hours), after an update of the SAXESS software. According to the news the problem is that no one can login.

    (This is the longest stop since March 1999 when it was closed for a whole day. Reason? Deployment of SAXESS)

    So it would be fair to say that the new software was no great saxess?

    Thank you! I'll be here all week!

  • Reverend (unregistered) in reply to powerlord
    powerlord:
    It could be worse... it could be Schrödinger's cat!

    The cat is immortal as long as you never look at it.

  • Bill O'Reilly (unregistered)

    F'ing think sucks! We're doing in it live!

    Captcha: There aren't any words there. What does that even mean?

  • Walleye (unregistered) in reply to matt
    matt:
    NaN:
    taylonr:
    Any objections raised by the developers were met with the same reaction you’d get from a dog after explaining the Pythagorean Theorem – blank, drooling faces.

    You have a pretty stupid dog

    Yeah, my dog does logarithms now.

    I'm having problems getting it to learn English though, so he can't tell me his answers.

    My dog is awesome.

    I ask my dog what the derivative of y = (5ln(4x) + y root(10) - 5 * 10010^(5x)) * (9 - (33)) is, and he says nothing.

    Beat that.

    My dog is working on a Unified Field Theory. He's out in the backyard digging holes. I think he's building a supercollider.

  • (cs) in reply to Zylon
    Zylon:
    AdT:
    Wait... they had a separate development environment? That's not following proper developmestuction procedure.
    For the love of god, just let that awful neologism die.
    I prefer Fowler's description of such syllabic agglutinations as "a nonce-word." That should put people off.

    For what it's worth, I hate the word "neologism" as well...

  • Hans (unregistered) in reply to Nicholas
    Nicholas:
    You know, the real problem in this scenario is with the lousy developers. They should just frickin' get it right the first time. I don't know how many hundreds of people I've met who really just can't be trusted to code. Use your brains, people! Don't call it done if it isn't done! Think about *all* the code paths! Learn the system you're working with. And don't whine about not having the time.

    Because when you get two hours for a new feature in a code base you have never seen before, you obviously should work through the night and the weekend just to get all of this done. Yes.

    Just remember this mantra, which has always served me well: no matter how much you think it's somebody else's fault, it's really your fault. And your problem. You have no idea the world of problems this attitude will fix, and how far it will take you.

    To a shitty deskjob where you always gets to solve other peoples problems? Or am I underestimating the situation now?

    Though it may just make you frustrated with everybody else's incompetence.

    I find I can be like that without also solving their problems...

  • Bill O'Reilly (unregistered) in reply to dpm
    Outlaw Programmer:
    Eventually, the company motto became, "We'll test it in production!"

    WE'LL DO IT LIVE!

  • jk (unregistered) in reply to Outlaw Programmer
    Outlaw Programmer:
    JC:
    John:
    And - most places I've worked at have done this, at least to some extent.

    Financial institutions really ought to know better though

    Financial institutions seem to be the worst for this sort of stuff in my experience. I used to work at Lehman Brothers and we had several similar instances caused by lack of testing and a general, we need it in there ASAP attitude.

    This mirrors my (limited) Wall St. experience, too. Everyone in management hated the idea of testing. Most clients just could not grasp the concept of testing. Yes, you're buying extra hardware for the QA layer but you will save thousands (millions?) in the long run, dammit! Clients that actually did buy servers for QA would often neglect them so that they became wildly out of sync with their production systems.

    Eventually, the company motto became, "We'll test it in production!"

    I had the opposite experience at a large financial firm. No names, but the ceo was canned for non-performance, only allowed to take home $100 Million in severance. Anyway, they had a six-sigma process that required change requests to be properly formatted (paper only), reviewed, approved by three levels of management, then developed, integrated in testing, tested, UAT, deployment tested, and deployed. Spent four years developing a shell replacement for Windows NT 4 workstation that would include every function possibly needed by anyone, and no other function allowed. By the time they were ready to roll out, NT4 desktops had been replaced by XP. Good times.

    Is there a happy medium between no testing and nothing but testing?

  • (cs) in reply to Reverend
    Reverend:
    powerlord:
    It could be worse... it could be Schrödinger's cat!

    The cat is immortal as long as you never look at it.

    Ummm ... no it's not.

    Although that does bring up a possibility that Schrödinger never considered. For some reason, he was only interested in an either/or.

    How would he know, upon opening the box, that the dead cat in front of him expired because of the state of the relevant sub-atomic particle, rather than just, say, spontaneously combusting through some bizarre internal variant on cold fusion?

    I mean, I haven't counted them, but ... there's a whole heck of a lot of sub-atomic particles in yer average cat.

  • (cs) in reply to Bill O'Reilly
    Bill O'Reilly:
    Outlaw Programmer:
    Eventually, the company motto became, "We'll test it in production!"
    WE'LL DO IT LIVE!
    Why don't we d-do it in the Prod? Why don't we do it in the Prod? Well, no one will be testing it... Why don't we do it in the Prod?
  • Synchronos (unregistered) in reply to powerlord
    powerlord:
    It could be worse... it could be Schrödinger's cat!

    Or then again, it could be not. You never know before it's too late.

  • Synchronos (unregistered) in reply to NaN
    NaN:
    Yeah, my dog does logarithms now.

    I'm having problems getting it to learn English though, so he can't tell me his answers.

    Times are tough, huh, bud?

  • Archibald Buttocks (unregistered) in reply to WallStreetDeveloper
    WallStreetDeveloper:
    I was surprised he said the pay was dire. Where else can you earn $200k+ as a developer?

    Not developing trading software! Maybe it's different in the UK, but the highest paid guys in the company were getting $80k, tops.

Leave a comment on “Deploy! Deploy! Deploy!”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article