• Steve (unregistered)

    Anyone can be first if they don't take the time to think of anything logical to write.

  • MaxPolun (unregistered)

    Promoted so they can't do any more damage... good choice

  • (cs) in reply to MaxPolun

    well it's always good to add testing code in prod, especially when you haven't added it to dev and staging as well [:|]

  • synonymouscowherd (unregistered) 
    <font>IF</font> (<font>CASTRAND</font>() * 100 + 1 <font>AS INT</font>) = 7)

    missing a '(' aren't you?
  • (cs)
    Alex Papadimoulis:
    <font color="#000099">... turns out that a DBA had inserted that code on the production (but not in the development or QA) database for "testing" purposes and forgotten to remove it. But, fortunately, that particular DBA was promoted to a lead position, and wouldn't be "down in the code" as much.</font><font color="#000099">


    So some Paulas get promoted.

    "lead"?  As in heavy stuff that makes you sink in water?  No such luck?  Drat!

    Sincerely,

    Gene Wirchenko

    </font>
  • britt (unregistered)

    So let's randomly roll back transactions?  IN PRODUCTION???  Talk about Test First(tm).

  • Theo (unregistered)

    editing live code is so much fun!!!  i once ssh'ed into a production webserver, modifieid some code hastily, escaped :wq and exited.  i reloaded the page and, wham!  website destroyed!!!!!  man i got email ever so quickly....and funny thing is, the site wasnt even used that much.....typical  

  • (cs) in reply to britt

    Not 6, 7!! 7 chipmunks twirlin' on a branch, eatin' lots of sunflowers on my uncle's ranch. You know that old children's tale from the sea.

    Step into my office!

  • (cs) in reply to Gene Wirchenko

    Gene Wirchenko:
    Alex Papadimoulis:
    <FONT color=#000099>... turns out that a DBA had inserted that code on the production (but not in the development or QA) database for "testing" purposes and forgotten to remove it. But, fortunately, that particular DBA was promoted to a lead position, and wouldn't be "down in the code" as much.</FONT><FONT color=#000099>


    So some Paulas get promoted.

    "lead"?  As in heavy stuff that makes you sink in water?  No such luck?  Drat!

    Sincerely,

    Gene Wirchenko

    </FONT>

    "Lead" as in the stuff in the paint chips that Paulas seem to have consumed as children.

  • (cs) in reply to Manni

    Manni:
    Not 6, 7!! 7 chipmunks twirlin' on a branch, eatin' lots of sunflowers on my uncle's ranch. You know that old children's tale from the sea.

    Step into my office!

    Seven is a number with great significance:

    There are 7 days in a week,

    The lunar cycle is divisible by 7 (28 days),

    there were 7  Dwarfs,

    et cetera, et cetera

  • (cs)

    So let's see if I got this right, every one out of ~100 transactions are randomly rolled back? What purpose could that possibly serve even in a testing environment? The only thing I can think of is to test an application's error handling capabilities by randomly sending it an error. But still, why use a deadlock exception? How about "RAISEERROR( 'This transaction has been randomly nuked ... please try again' ..." instead.

  • (cs)

    This company apparently uses the opposite of the "cream rises to the top" method for promoting people from within.  They instead believe that shit floats, and therefore should be moved up.

  • (cs)

    I'd be more interested in what the DBA was trying to test... was he unconvinced that ROLLBACK TRANSACTION was implemented properly by the vendor?!?  And, what, he wanted to be sure it would work if executed at random intervals only?!?

    Nah, this is kind of a weak WTF... stupid test code gets left in prod code all the time no matter where you work.  It shouldn't happen, but it does.  The WTF is what the code was meant for in the first place??

  • nimrand (unregistered)

    WTF does not do this justice.  We need an acronym with a lot more cursing.

  • (cs)

    Promoted!

    Ah, The Peter Principle at its finest.

    I used to enjoy reading the regular column Dr. Laurence J.Peter wrote for Psychology Today magazine.

  • (cs)
    Alex Papadimoulis:
    <font color="#000099">IF</font> (<font color="#000099">CAST<font color="#000099">RAND</font>() * 100 + 1 <font color="#000099">AS INT</font>) = 7)
<font color="#000099">BEGIN
  RAISERROR</font>( <font color="#990000">'Transaction (Process ID %d) was deadlocked on %.*ls resources '</font> +
             <font color="#990000">'with another process and has been chosen as the deadlock victim. '</font> +
             <font color="#990000">'Rerun the transaction.'</font>, 13, 10, @@SPID, 8, <font color="#990000">'lock'</font> )
  <font color="#000099">ROLLBACK TRANSACTION
	RETURN
END</font></font>
    <font color="#000099">

    The easy fix is, of course, to change that to CASTRAND() * 10000.  That way you will only have to roll back 1 in every 10,000 transactions, which is a much more manageable number.
    </font>
  • (cs) in reply to marvin_rabbit

    This is a weird one.  I cannot fathom what they are testing by manually raising errors masquerading as deadlocks.  Are they trying to be sure that the database and application code is written to gracefully handle all possible deadlocks and/or rolled back transactions?  Why not actually trap real deadlocks and consider actually troubleshooting real ones and writing code that avoids them?  

    I just can't comprehend the thought process behind this one, even more than the usual WTF's we see here from day to day.

  • (cs) in reply to Jeff S
    Jeff S:
    This is a weird one.  I cannot fathom what they are testing by manually raising errors masquerading as deadlocks.  Are they trying to be sure that the database and application code is written to gracefully handle all possible deadlocks and/or rolled back transactions?  Why not actually trap *real* deadlocks and consider actually troubleshooting real ones and writing code that *avoids* them?  

    I just can't comprehend the thought process behind this one, even more than the usual WTF's we see here from day to day.


    In a well written application, it's pretty difficult to deliberately cause a real deadlock. But it can happen. For that reason, it can make some sense to generate artificial error messages like given in the WTF to check that the deadlock handling code is working correctly. That said, I would rather insert code that always throws the error, test the error handling code, and remove the artificial error creation code after that.
  • Ann Coulter (unregistered) in reply to ParkinT
    ParkinT:

    Promoted!

    Ah, The Peter Principle at its finest.

    I used to enjoy reading the regular column Dr. Laurence J.Peter wrote for Psychology Today magazine.



    Let's hope that it's an example of the reverse Peter Principle where the incompetent are promoted to a level of competence.

  • (cs)
    Alex Papadimoulis:

    Stuart's team poured through the code



    I can see it now... coders digitised in some kind of transporter, turned into liquid binary and poured through the code... of course, I'd even argue that you can't pore "through" something, but you can pore over something.

    Funny story.. Cisco once sent me a proposal - written by some young pup of a pre-sales engineer, who talked about "...on a 45MB back-hall..."

    laughed my pedantic socks off so I did...

  • (cs) in reply to versatilia

    The +1 is completely unneccesary, even if Rand gives you a number starting at zero.

  • (cs) in reply to versatilia

    Wait!  You mean DBA's can get promoted?  And this is how to do it??  Man, I am gonna write me into the corner office this afternoon......

  • (cs) in reply to versatilia
    versatilia:
    Alex Papadimoulis:

    Stuart's team poured through the code



    I can see it now... coders digitised in some kind of transporter, turned into liquid binary and poured through the code... of course, I'd even argue that you can't pore "through" something, but you can pore over something.

    Funny story.. Cisco once sent me a proposal - written by some young pup of a pre-sales engineer, who talked about "...on a 45MB back-hall..."

    laughed my pedantic socks off so I did...

    That is funny!

    You have hit on a "sore point" of mine.  There are many criminal abuses asserted on our language (well, I guess it is YOUR language - being from the UK) and I find myself cringing often when they surface.

    I have mentioned before on this forum how comical I find it to read software developer positions advertised in the local paper that require:

    "experience in Pearl and Sequel"

  • WTFer (unregistered) in reply to marvin_rabbit

    Developing your own error logging/notification system is a WTF. "not invented here" / Yet Another Error Logging System. Although that DBA should be fired for trying stupid stuff like that on a PRODUCTION machine.

  • (cs) in reply to marvin_rabbit
    marvin_rabbit:
    Alex Papadimoulis:
    <FONT color=#000099>IF</FONT> (<FONT color=#000099>CAST<FONT color=#000099>RAND</FONT>() * 100 + 1 <FONT color=#000099>AS INT</FONT>) = 7)
<FONT color=#000099>BEGIN
  RAISERROR</FONT>( <FONT color=#990000>'Transaction (Process ID %d) was deadlocked on %.*ls resources '</FONT> +
             <FONT color=#990000>'with another process and has been chosen as the deadlock victim. '</FONT> +
             <FONT color=#990000>'Rerun the transaction.'</FONT>, 13, 10, @@SPID, 8, <FONT color=#990000>'lock'</FONT> )
  <FONT color=#000099>ROLLBACK TRANSACTION
	RETURN
END</FONT></FONT>

    <FONT color=#000099>


    The easy fix is, of course, to change that to CASTRAND() * 10000.  That way you will only have to roll back 1 in every 10,000 transactions, which is a much more manageable number.
    </FONT>

    LOL...

    seriously, I can't imagine why you would put such message and roll back the transaction randomly

    reminds me of some co-workers who struggled for days trying to find a NULL in some field, the queries just wouldn't work as expected... until someone decided to try searching for 'NULL'

  • Steve (unregistered)

    This looks to me like a true -3 on the Capability Im-Maturity Model http://www.stsc.hill.af.mil/crosstalk/1996/11/xt96d11h.asp (Undermining / Sabatoge).  No matter how hard I try to find a valid reason for this I cannot make sense out of it.

  • boohiss (unregistered)

    Oh, cmon, like none of you have ever edited anything in production? This is a nasty one but get off your high horse will you!

  • boohiss (unregistered) in reply to ParkinT

    You think that might have something to do with the person on the other end of the phone writing down exactly what they hear?

  • (cs) in reply to boohiss
    Anonymous:
    Oh, cmon, like none of you have ever edited anything in production? This is a nasty one but get off your high horse will you!


    Not like that, no.

    I have had to edit data, because of a failure, but changing program behaviour on a production system without testing first?  No.

    Sincerely,

    Gene Wirchenko

  • (cs) in reply to Steve
    Anonymous:
    This looks to me like a true -3 on the Capability Im-Maturity Model http://www.stsc.hill.af.mil/crosstalk/1996/11/xt96d11h.asp (Undermining / Sabatoge).  No matter how hard I try to find a valid reason for this I cannot make sense out of it.


    A great read!  Thank you for the pointer.

    Sincerely,

    Gene Wirchenko

  • diaphanein (unregistered) in reply to boohiss

    Anonymous:
    Oh, cmon, like none of you have ever edited anything in production? This is a nasty one but get off your high horse will you!

    I once hex-editted a binary in production in order to fix an Sql query that was failing.  Thank goodness the fix was to delete chars (well, overwrite with spaces), instead of adding them...

  • (cs) in reply to Gene Wirchenko

    Gene Wirchenko:
    Anonymous:
    Oh, cmon, like none of you have ever edited anything in production? This is a nasty one but get off your high horse will you!


    Not like that, no.

    I have had to edit data, because of a failure, but changing program behaviour on a production system without testing first?  No.

    Sincerely,

    Gene Wirchenko

    PFFFFTTT!!!...

    newbies!!

  • (cs) in reply to diaphanein
    Anonymous:

    Anonymous:
    Oh, cmon, like none of you have ever edited anything in production? This is a nasty one but get off your high horse will you!

    I once hex-editted a binary in production in order to fix an Sql query that was failing.  Thank goodness the fix was to delete chars (well, overwrite with spaces), instead of adding them...

    been there, done that[8-|]

  • DanixDefcon5 (unregistered) in reply to diaphanein

    Now, surely you haven't heard about something as ugly as doing a blooper on a Production database ... in MySQL, using MyISAM tables. Oops ... no ROLLBACK!

    Non-Transactional Databases are the worst WTF ever concieved, except maybe for XML-powered databases (I don't mean xml-supporting DBMS, I mean DBMS that actually STORE the data in XML).

    And yes, I know there is InnoDB for MySQL, but still, they didn't just stick in Transactional stuff in the beginning, they actually promoted non-transactional production systems when they started up!!! However, it seems they have vaporized such remarks without a trace from their documentation.Super Angry [8o|]

  • (cs) in reply to limelight
    limelight:
    So let's see if I got this right, every one out of ~100 transactions are randomly rolled back? What purpose could that possibly serve even in a testing environment? The only thing I can think of is to test an application's error handling capabilities by randomly sending it an error. But still, why use a deadlock exception? How about "RAISEERROR( 'This transaction has been randomly nuked ... please try again' ..." instead.


    Yea, that's the WTF. With that wording, how would you be able to differentiate the actual errors from the one's you're generating by yourself? Otherwise, I'm sure you could come up with reasons that you'd need to test an application and make sure it can do a bunch of rollbacks while under a decent load...You should always make sure your test code isn't left lying around though. That many errors should never have gotten through any sort of quality process.
  • (cs) in reply to Satanicpuppy
    Satanicpuppy:
    limelight:
    So let's see if I got this right, every one out of ~100 transactions are randomly rolled back? What purpose could that possibly serve even in a testing environment? The only thing I can think of is to test an application's error handling capabilities by randomly sending it an error. But still, why use a deadlock exception? How about "RAISEERROR( 'This transaction has been randomly nuked ... please try again' ..." instead.


    Yea, that's the WTF. With that wording, how would you be able to differentiate the actual errors from the one's you're generating by yourself? Otherwise, I'm sure you could come up with reasons that you'd need to test an application and make sure it can do a bunch of rollbacks while under a decent load...You should always make sure your test code isn't left lying around though. That many errors should never have gotten through any sort of quality process.


    Deadlocks are probably handled differently than all other errors. It's quite reasonable to simply retry a transaction that failed because of a deadlock, probably even without reporting the problem to the end-user.
  • (cs) in reply to Satanicpuppy
    Satanicpuppy:
    limelight:
    So let's see if I got this right, every one out of ~100 transactions are randomly rolled back? What purpose could that possibly serve even in a testing environment? The only thing I can think of is to test an application's error handling capabilities by randomly sending it an error. But still, why use a deadlock exception? How about "RAISEERROR( 'This transaction has been randomly nuked ... please try again' ..." instead.


    Yea, that's the WTF. With that wording, how would you be able to differentiate the actual errors from the one's you're generating by yourself? Otherwise, I'm sure you could come up with reasons that you'd need to test an application and make sure it can do a bunch of rollbacks while under a decent load...You should always make sure your test code isn't left lying around though. That many errors should never have gotten through any sort of quality process.

    Maybe they had to do it that way because some code somewhere parses the error text to figure out what has gone wrong - wouldn't surprise me at all, not around here anyway...

    (Or maybe it's just sabotage/criminal stupidity.)

  • (cs) in reply to BlackTigerX
    BlackTigerX:

    Gene Wirchenko:
    Anonymous:
    Oh, cmon, like none of you have ever edited anything in production? This is a nasty one but get off your high horse will you!


    Not like that, no.

    I have had to edit data, because of a failure, but changing program behaviour on a production system without testing first?  No.

    Sincerely,

    Gene Wirchenko

    PFFFFTTT!!!...

    newbies!!



    Yea, I've had to shoot from the hip a few times myself as well. That's probably about the most exciting (though not necessarily exciting in a good way) part of a programmers job. The business I'm in right now is SO time and schedule driven that stuff has to be flipped on the fly WAY more often than I would like. Makes your heart beat fast.
  • nsayn (unregistered)

    You can't tell me that nobody grepped the code for that error message to find out where it was coming from. And when it didn't appear in dev or test, grep the production code as well. WTF?

    Maybe that was their standard 'deadlock' error message, and could have come from several places in the code. In that case, why didn't their 'standard' deadlock error message report the file and line in the code that it came from? WTF?

    Sounds like the development and debugging practices of the team as a whole are a bit special.


  • (cs) in reply to makomk

    It looks like he injected the random error so that he could take credit for removing the "problem" later on.

  • Anita Tinkle (unregistered) in reply to Steve

    I've worked for a couple of -1 organizations (as we have all), which I think is the norm for most of the small IT shops.  I can't imagine how bad a -3 organization is (indicitive of a medium-to-large organization that has a long lifetime).

    You hear the words "burn out" often and you also see lots of people quit (enough that there's a job posting for an IT position up on the monsterboard to replace someone who left at least 1x a quarter) to go to greener pastures in a -1.

    I wonder how this guy managed to stay at a -3 organization?

    Anonymous:
    This looks to me like a true -3 on the Capability Im-Maturity Model http://www.stsc.hill.af.mil/crosstalk/1996/11/xt96d11h.asp (Undermining / Sabatoge).  No matter how hard I try to find a valid reason for this I cannot make sense out of it.

  • (cs) in reply to Satanicpuppy
    Satanicpuppy:
    BlackTigerX:

    Gene Wirchenko:
    Anonymous:
    Oh, cmon, like none of you have ever edited anything in production? This is a nasty one but get off your high horse will you!


    Not like that, no.

    I have had to edit data, because of a failure, but changing program behaviour on a production system without testing first?  No.

    Sincerely,

    Gene Wirchenko

    PFFFFTTT!!!...

    newbies!!



    Yea, I've had to shoot from the hip a few times myself as well. That's probably about the most exciting (though not necessarily exciting in a good way) part of a programmers job. The business I'm in right now is SO time and schedule driven that stuff has to be flipped on the fly WAY more often than I would like. Makes your heart beat fast.


    Sometimes a change is so urgent that there is no other chance than a shot from the hip. But if that happens too often, it's a sign of a bad (or non-existing) management. The most likely case of such an urgent modification is when a change was made that was not good, eighter because it wasn't proberly tested or because the specification for that change was wrong. Some week ago, a client demanded "prevent to program from doing X". I asked "no exceptions from that rule?". They: "No". I: "Are you sure about that? Think of (common exceptions at other clients).".
    They: "No".

    Guess.

  • Anita Tinkle (unregistered) in reply to nsayn

    Tell me, without going to the system tables in SQL Server or running a cursor to call sp_helptext on all the stored procs, how do you quickly "grep" TSQL stored procs?

    That's right.  You can't.  You have to write one or query the underlying schema manually since it's not intuitively easy.

    USE [dbname]
    SELECT SPECIFIC_NAME, ROUTINE_DEFINITION
    FROM INFORMATION_SCHEMA.ROUTINES
    WHERE
    ROUTINE_DEFINITION LIKE '%foo%'


    Anonymous:
    You can't tell me that nobody grepped the code for that error message to find out where it was coming from. And when it didn't appear in dev or test, grep the production code as well. WTF?

    Maybe that was their standard 'deadlock' error message, and could have come from several places in the code. In that case, why didn't their 'standard' deadlock error message report the file and line in the code that it came from? WTF?

    Sounds like the development and debugging practices of the team as a whole are a bit special.


  • Anita Tinkle (unregistered) in reply to Anita Tinkle

    (or you can script out all the stored procs from SQLEM)

  • (cs) in reply to BlackTigerX
    BlackTigerX:

    Gene Wirchenko:
    Anonymous:
    Oh, cmon, like none of you have ever edited anything in production? This is a nasty one but get off your high horse will you!


    Not like that, no.

    I have had to edit data, because of a failure, but changing program behaviour on a production system without testing first?  No.

    PFFFFTTT!!!...

    newbies!!


    Who?  Certainly not me.

    Sincerely,

    Gene Wirchenko

  • (cs) in reply to Anita Tinkle

    Select     object_name(id)
    from syscomments
    where text like '%foo%'

  • (cs) in reply to Ken

    Ken:
    Select     object_name(id)
    from syscomments
    where text like '%foo%'

    Stupid forum

  • (cs) in reply to nsayn
    Anonymous:
    You can't tell me that nobody grepped the code for that error message to find out where it was coming from. And when it didn't appear in dev or test, grep the production code as well. WTF?

    Maybe that was their standard 'deadlock' error message, and could have come from several places in the code. In that case, why didn't their 'standard' deadlock error message report the file and line in the code that it came from? WTF?

    Sounds like the development and debugging practices of the team as a whole are a bit special.




    I guess this is the standard error message coming from the database system. You do not normaly expect that the application fakes that. It's like grepping for "general protection failure" in the source of a Windows program.
  • (cs) in reply to ammoQ
    ammoQ:
    Jeff S:
    This is a weird one.  I cannot fathom what they are testing by manually raising errors masquerading as deadlocks.  Are they trying to be sure that the database and application code is written to gracefully handle all possible deadlocks and/or rolled back transactions?  Why not actually trap *real* deadlocks and consider actually troubleshooting real ones and writing code that *avoids* them?  

    I just can't comprehend the thought process behind this one, even more than the usual WTF's we see here from day to day.


    In a well written application, it's pretty difficult to deliberately cause a real deadlock. But it can happen. For that reason, it can make some sense to generate artificial error messages like given in the WTF to check that the deadlock handling code is working correctly. That said, I would rather insert code that always throws the error, test the error handling code, and remove the artificial error creation code after that.


    More importantly, if you insert temp code into a production system, then wrap it in "if user = the_consultant then { ... }" so that it doesn't affect the regular users.

  • anonymous (unregistered) in reply to limelight
    limelight:
    So let's see if I got this right, every one out of ~100 transactions are randomly rolled back? What purpose could that possibly serve even in a testing environment? The only thing I can think of is to test an application's error handling capabilities by randomly sending it an error. But still, why use a deadlock exception? How about "RAISEERROR( 'This transaction has been randomly nuked ... please try again' ..." instead.


    Because that would make sense, so it wouldn't be here, so you wouldn't have seen it, so you wouldn't have been able to complain about it and then we'd have a paradox and the universe would explode.

Leave a comment on “The Phantom Deadlock”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article