• Bob Philhower (unregistered)

    I'd like to see a list of the 17 times that solving the "meta-problem" was useful.

  • Glenn (unregistered) in reply to Bob Philhower

    Anonymous:
    I'd like to see a list of the 17 times that solving the "meta-problem" was useful.

    Unfortunately, they all occurred in 1959, and the wired patch panels were eventually recycled. 

  • Kluge Doctor (unregistered) in reply to Glenn
    Anonymous:

    Anonymous:
    I'd like to see a list of the 17 times that solving the "meta-problem" was useful.

    Unfortunately, they all occurred in 1959, and the wired patch panels were eventually recycled. 

    No, they are called technology break throughs, like going from  VHS to DVD.

  • ewhac (unregistered)

    "Hey!  I remember this thing from my undergraduate week when we skimmed over Doug Comer's XINU book.  I think they were called 'metaphores'..."

    Bloody amateurs...

  • (cs)
    Alex Papadimoulis:

    His transactional system was built on top of SQL Server 2000 and involved using a table called db__locks which had a row for each table in the database and contained the table's name and a lock level of 0 (not locked) or 1 (exclusively locked). Obviously, he had big expansion plans for the future.

    Ok, smart Alec. You really killed the fun before they started on it.

    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock 

    Alex Papadimoulis:

    Fun fact: this transactional system comes with an admin utility to reset the db__Locks table if needed.

    Another fun fact: on-call developers use this utility fairly often in the middle of the night when they're woken up by a support call

    Those on-call developers may be script readers. Over here, IT support staff are called engineers, mere programmers are called users, engineers are called consultants........

    Reminds me of Assenture and their thousand strong team full of architects.

    Anyway, Rob's predecessor may not have created that fun utility. That's usually the sign of someone continuing to use the virtual lock without understanding the WTF nature of it, or maybe management was Chocoboed into believing that the virtual lock was really useful.

  • HatTrick1914 (unregistered) in reply to ewhac
    Anonymous:

    "Hey!  I remember this thing from my undergraduate week when we skimmed over Doug Comer's XINU book.  I think they were called 'metaphores'..."

    Bloody amateurs...

    Oh they may not be amateurs, I worked for a company that had an off the shelf quality monitoring system that did the exact same thing. It was developed by people that had 20+ years experience each. And if that wasn't bad enough they also had no clue what normalized data was, everything was stored in a flat table with humdreds of fields. After years of expansion they reached the limits of the number of fields you could have in a single table.

  • Fruny (unregistered) in reply to kuroshin
    kuroshin:
    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock

    lock level 5 = record file not found
  • (cs)

    Anonymous:
    Inner-Platform Effect is also known (perhaps more correctly) as Second-System Effect: http://en.wikipedia.org/wiki/Second_system_syndrome

    If I'm not wrong, I'm working on reversing that effect in one of our systems here.

    Too bad, I've got to hide all the ugly stuff instead of refactoring it or writing from scratch.

  • (cs)

    OMG!

    One of the fundamental aspects of the implementation of any locking strategy is that the "discover the item is not currently locked - lock it ourselves" bit has to be atomic... otherwise two locks can get set at the same time!

    This was doomed to failure from the start (and as pointed out... pointless!!) 

  • (cs)

    Sweet jesus. If I was this guy's employer, I'd not only fire him, but sue him for the wages they paid him for deceiving the company into believing that he was competant. Then I'd fire whoever hired this guy, because they're clearly not competant at decision making

  • (cs) in reply to Fruny
    Anonymous:
    kuroshin:
    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock

    lock level 5 = record file not found

    lock level 6 = headlock

    lock level 7 = cockb-lock
     

  • (cs)

    Lovely race condition there.

    Next they have to implement message queues and events via database tables to further increase serialization.

  • (cs)

    Alex Papadimoulis:
    * I would have linked to the Wikipedia for these anti-patterns, but for some reason, there's no article on them. I assume everyone was just too busy contributing to the thirteen-page article dissertation on the Chocobo, the fictional bird from the Final Fantasy video game series.



    What? No love for the Chocobo? For shame, Alex. 

  • (cs) in reply to Fruny
    Anonymous:
    kuroshin:
    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock

    lock level 5 = record file not found

    Uh wait, I forgot the most important ones :

    lock level -1 : locklock [ A lock on an existing lock, smarty ]

    lock level -2 : recursivelock [ Surely, something enterprisey has to have recursion ]
  • (cs) in reply to kuroshin
    kuroshin:
    Anonymous:
    kuroshin:
    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock

    lock level 5 = record file not found

    Uh wait, I forgot the most important ones :

    lock level -1 : locklock [ A lock on an existing lock, smarty ]

    lock level -2 : recursivelock [ Surely, something enterprisey has to have recursion ]

     

    lock level -3: powerlock [A lock that cannot be broken using the admin utility, must be broken by a level 10 DBA with 18 dexterity. Roll for damage.]

  • (cs) in reply to kuroshin

    The best part is that the locking logic itself is not transactional, so it's entirely possible for two or more processes to end up with the lock on a table.  Fun!

  • (cs) in reply to Colin McGuigan

    Colin McGuigan:
    The best part is that the locking logic itself is not transactional, so it's entirely possible for two or more processes to end up with the lock on a table.  Fun!

    You've pointed out the uselessiness (like truthiness) of the admin utility. The locks are stored in a table that can be locked.

  • (cs) in reply to ewhac
    Anonymous:

    "Hey!  I remember this thing from my undergraduate week when we skimmed over Doug Comer's XINU book.  I think they were called 'metaphores'..."

    Bloody amateurs...

     Semaphores???

  • (cs) in reply to kuroshin
    kuroshin:

    Colin McGuigan:
    The best part is that the locking logic itself is not transactional, so it's entirely possible for two or more processes to end up with the lock on a table.  Fun!

    You've pointed out the uselessiness (like truthiness) of the admin utility. The locks are stored in a table that can be locked.



    You can lock the lock table using this locking scheme, but the only thing enforcing that lock is the application code.  So the admin utility simply doesn't check to see if the lock table is locked before unlocking some other table.

    Try saying that three times fast.
  • Cody (unregistered) in reply to kuroshin

    Colin McGuigan:
    The best part is that the locking logic itself is not transactional, so it's entirely possible for two or more processes to end up with the lock on a table.  Fun!

    That was fixed in the update.  Now the lock table is locked when locking a table.
  • (cs) in reply to DigitalLogic

    I've wrapped built-in functionality to suit my needs. But Completely re-invent it?

    Imagine what it would look like if he wanted to get to the record-locking level?

  • (cs)

    Anonymous:
    Inner-Platform Effect is also known (perhaps more correctly) as Second-System Effect: http://en.wikipedia.org/wiki/Second_system_syndrome

     That's not the same as inner-platform. I think there should be a wikipedia entry about inner-platform.

  • (cs)

    Alex Papadimoulis:
    ...meta-problem: a problem with the process of creating a solution for the actual problem.

    The following forum post on Joels is already a classic when it comes to describe how silly meta problem solving is, a must read: Why I Hate Frameworks


     

     

  • (cs) in reply to GettinSadda
    GettinSadda:

    OMG!

    One of the fundamental aspects of the implementation of any locking strategy is that the "discover the item is not currently locked - lock it ourselves" bit has to be atomic... otherwise two locks can get set at the same time!

    This was doomed to failure from the start (and as pointed out... pointless!!) 





    <sarcasm>
    That's an easy fix.


    -- Wait for lock
    SET TRANSACTION ISOLATION LEVEL SERIALIZABLE

    WHILE @LockLevel <> '0'
    BEGIN
        BEGIN TRANSACTION
        SELECT @LockLevel = LockLevel
        FROM db__Locks
        WHERE LockTbl = 'GL_Operations'

        IF @LockLevel <> '0' ROLLBACK TRANSACTION
    END

    -- Set lock
    UPDATE db__Locks
    SET LockLevel = '1'
    WHERE LockTbl = 'GL_Operations'
    COMMIT TRANSACTION

    ... snip whole bunch of code ...

    -- Clear lock
    UPDATE db__Locks
    SET LockLevel = '0'
    WHERE LockTbl = 'GL_Operations'


    P.S - The real WTF is that he didn't put the locking logic in stored procedures.  That would make adding the above fix much easier to implement.

    </sarcasm>
  • Tango Uniform (unregistered) in reply to DigitalLogic

    Here is my question relating to database locks.

    1. User selects a record in a GUI list to edit
    2. System loads data into GUI edit window
    3. 30 minutes go by (while user is editing data in GUI, anyone else trying to edit it gets a message it is locked)
    4. User hits save button
    5. Record in database is updated

    What sort of "locking" mechanism would you use to implement this type of transaction, where a record has been read to be edited by a user, but this might take quite some time?  How would having a limited pool of database connections impact such an implementation?  How would the need to have a portable (between different databases) affect the implementation?

    Basically, I have seen designs that have some sort of "lock" record on tables to indicate the record is in use by some user.  This is not done necessarily to reimplement the concept of a transaction.  In fact, these systems use a database transaction to test and update the lock field to ensure only one user gets the lock.  It is more used for locking a record or set of records in order to allow a user to look at and edit the data, when a pessimistic lock is required.

    In addition, how would one implement a portable optimistic locking scheme.  Most of those I have seen involve having a "last updated" timestamp, where ensuring it hasn't changed is part of the update sql.

    In essense, I have seen a lock flag field used for pessimistic locking, and a timestamp field used for optimistic locking, and have not seen any problems with these techniques.  They seem to be versatile and portable, and don't require holding a database connection for long periods of time while the record is being edited on a GUI.

    Thanks. 

  • (cs)

    After reading this story, all I have to say to Robert Rossney is:

     

    Good Lock !

  • Craig (unregistered)

    "Now before all you database developers scream "Use Built-in Transactions!," let me remind you that there are a whole lot of problems associated with built-in database transactions"

    But what about when built-in transactions are insufficient?

    Sybase IQ supports transactions, but only allows one 'writer' thread per table.  This means that if you want to have multiple ways to update a single table you have to 'single-stream' them.

  • Milkshake (unregistered) in reply to rmg66
    rmg66:
    Anonymous:

    "Hey!  I remember this thing from my undergraduate week when we skimmed over Doug Comer's XINU book.  I think they were called 'metaphores'..."

    Bloody amateurs...

     Semaphores???

    Give him a break.  He only skimmed over the book, but obviously that's quite enough to justify him in calling someone else an amateur.

  • (cs)

    wtf?! sounds like the weed was smoked a little too much that day.

    use a rdbms and then add layer upon layer of "management" constraints aka *slow* the system down.....

     You Geek Code Something

  • Grog (unregistered)

    Why didn't he use a cursor to iterate over the list of locks? Come on, man!!!

  • Local guy (unregistered) in reply to Ghost Ware Wizard

    Nice to see some code :)

  • (cs) in reply to Tango Uniform
    Anonymous:

    Here is my question relating to database locks.

    1. User selects a record in a GUI list to edit
    2. System loads data into GUI edit window
    3. 30 minutes go by (while user is editing data in GUI, anyone else trying to edit it gets a message it is locked)
    4. User hits save button
    5. Record in database is updated

    What sort of "locking" mechanism would you use to implement this type of transaction, where a record has been read to be edited by a user, but this might take quite some time?  How would having a limited pool of database connections impact such an implementation?  How would the need to have a portable (between different databases) affect the implementation?

    Basically, I have seen designs that have some sort of "lock" record on tables to indicate the record is in use by some user.  This is not done necessarily to reimplement the concept of a transaction.  In fact, these systems use a database transaction to test and update the lock field to ensure only one user gets the lock.  It is more used for locking a record or set of records in order to allow a user to look at and edit the data, when a pessimistic lock is required.

    In addition, how would one implement a portable optimistic locking scheme.  Most of those I have seen involve having a "last updated" timestamp, where ensuring it hasn't changed is part of the update sql.

    In essense, I have seen a lock flag field used for pessimistic locking, and a timestamp field used for optimistic locking, and have not seen any problems with these techniques.  They seem to be versatile and portable, and don't require holding a database connection for long periods of time while the record is being edited on a GUI.

    Thanks. 

    The mechanism varies by database, so you would put this in your database-specific layer. In Oracle you would select the system change number pseudo-column, then check the SCN again in the where clause of your update statement. 

     

  • aliethel (unregistered)

    I have to ask what does that say about me that I actually read the entire article and enjoyed it?

  • anon (unregistered)

    Two lines, one platform,  one bridge:

    || P ^^
    || L ||
    || A BRIDGE -> Parking lot
    || T ||
    || F ||
    || O ||
    || R ||
    vv M ||

  • Ripper the Non-Believer (unregistered) in reply to John Bigboote

    Next week:the non-blocking chocoblock.

  • SwordfishBob (unregistered) in reply to anon
    Anonymous:
    Two lines, one platform,  one bridge:

    || P ^^
    || L ||
    || A BRIDGE -> Parking lot
    || T ||
    || F ||
    || O ||
    || R ||
    vv M ||

    Yeah, I recall seeing a few platforms as wobbly as variable-pitch-text-drawings, and with the end falling off..
  • iboB (unregistered) in reply to John Bigboote
    John Bigboote:
    kuroshin:
    Anonymous:
    kuroshin:
    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock

    lock level 5 = record file not found

    Uh wait, I forgot the most important ones :

    lock level -1 : locklock [ A lock on an existing lock, smarty ]

    lock level -2 : recursivelock [ Surely, something enterprisey has to have recursion ]

     

    lock level -3: powerlock [A lock that cannot be broken using the admin utility, must be broken by a level 10 DBA with 18 dexterity. Roll for damage.]

    lock level 0xFFFFFFFF : division by zero

  • Adam B. (unregistered) in reply to GettinSadda
    GettinSadda:

    OMG!

    One of the fundamental aspects of the implementation of any locking strategy is that the "discover the item is not currently locked - lock it ourselves" bit has to be atomic... otherwise two locks can get set at the same time!

    This was doomed to failure from the start (and as pointed out... pointless!!) 

     

    Actually, you can do it atomically with JDBC (probably ODBC and other mechanisms as well).  Update lock_table set lock_value=1 where lock_value=0 and lock_name="whatever".  If the return value is 1, you got the lock, otherwise no dice. There's even a legitimate (if exceedingly rare) use for this particular WTF. If it's a distributed system and the cost of cleaning up if you find out that the default optimistic working strategy is extremely high, this can be faster by reducing the number of times that you have to clean up.  Of course, this also leads to loads of issues like having to do lock reclamation if a process hangs or dies, deadlock detection and who knows how many other issues.

  • Adam B. (unregistered) in reply to Adam B.

    Oh, forgot that in that case they probably also should have implemented read-locks unless the data is always written after reading it.

  • xcor057 (unregistered) in reply to kuroshin
    kuroshin:
    Anonymous:
    kuroshin:
    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock

    lock level 5 = record file not found

    Uh wait, I forgot the most important ones :

    lock level -1 : locklock [ A lock on an existing lock, smarty ]

    lock level -2 : recursivelock [ Surely, something enterprisey has to have recursion ]

     recursivelock - A lock that re-locks itself when unlocked?

  • Wiki Refugee (unregistered)

    Funnily enough, I found TDWTF through a link from that very page.

     

    Captcha: genius. I quite agree.

  • (cs)
    Anonymous:
    Alex Papadimoulis:

    <font size="-1">* I would have linked to the Wikipedia for these anti-patterns, but for some reason, there's no article on them. I assume everyone was just too busy contributing to the thirteen-page article dissertation on the Chocobo, the fictional bird from the Final Fantasy video game series.</font>

     

    Not only it holds  <font size="-1">thirteen pages, it also exists in ten additional languages.
    </font>

    <font size="+1">D</font>amn!  Nothing in Klingon yet.
  • (cs)

    The most annoying part of this thing is that they can afford a Real Database System, and they elect to use it like Glorified MySQL 3.x. Clearly, a case of more money than sense.

    And there's nice articles in WIkipedia on all these anti-patterns. Not Invented Here, Invented Here, But Let's Reinvent It Anyway, The Inner-Platform Effect and many others... Say what you want about Wikipedians' willingness to cover Boring Topics, but idiocies of computer science are not among the Boring Topics No One Bothers To Cover. =)

  • (cs) in reply to John Bigboote
    John Bigboote:
    Anonymous:
    kuroshin:
    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock

    lock level 5 = record file not found

    lock level 6 = headlock

    lock level 7 = cockb-lock
     

     

    lock level 8 = Matlock 

  • (cs) in reply to triso

    (On Chocobo article)

    triso:
     

    Not only it holds  <font size="-1">thirteen pages, it also exists in ten additional languages.
    </font>

    <font size="+1">D</font>amn!  Nothing in Klingon yet.

    Completely off-topic post to hopefully insert some rationality (?) and put a bit of a stop on further off-topic divergences:

    1. Klingon language, last I checked, didn't have word for Chocobos. Or, IIRC, birds in general.
    2. Also, last I checked, Klingon Wikipedia was left permanently in read-only mode. There weren't enough contributors, or something like that.

     

  • Anonymous (unregistered) in reply to Tango Uniform

    "In essense, I have seen a lock flag field used for pessimistic locking, and a timestamp field used for optimistic locking, and have not seen any problems with these techniques.  They seem to be versatile and portable, and don't require holding a database connection for long periods of time while the record is being edited on a GUI."

     

    You are not really "locking" the record.  In fact, these schemes _rely_ on the internal atomic locking mechanisms to work.  What you are really talking about is record _versioning_.  Better to use a discrete integer to represent version changes rather than timestamp.  However, you don't really need to do that either.  Using predicated updates (updates that use the old values in the criteria) to verify it hasn't changed while updating it. 

  • (cs) in reply to Milkshake
    Anonymous:
    rmg66:
    Anonymous:

    "Hey!  I remember this thing from my undergraduate week when we skimmed over Doug Comer's XINU book.  I think they were called 'metaphores'..."

    Bloody amateurs...

     Semaphores???

    Give him a break.  He only skimmed over the book, but obviously that's quite enough to justify him in calling someone else an amateur.

    Great job on totally missing the joke, both of you. Whoosh.

     

    And stop linking Second System for Inner Platform! They're totally different pathologies! Writing your own scripting language that translates your script to C++ and compiles, translating errors back into script, is an example of the latter (unless you're <font size="-1">Bjorne Stroustrup</font>), whereas rewriting it later to include all the features from every Perl and PHP module you can find (and accepting the syntaxes of several different languages) is the former.

  • (cs) in reply to Tango Uniform

    I think* the normal way to implement optimistic locking, is to include the original values in the where clause of the update. That way the update will not update any rows, if the rows have been changed since the client did read them. (Update returns the number of rows modified, so you can find out if this have happend, and then take action)

     
    If you REALLY need pessimistic locking, but don't have any limit on how long the client can keep the lock,

    just set the transaction to fail if it can't aquire the needed locks. That way the application can handle the problem, either by waiting for x seconds and the retrying, or by informing the user that the specified operation can't be done because someone else are using the database.

     

    Atleast that the way I would do it. I don't hope this queto end up at http://www.dbdebunk.com :}

     

     *(Not sure this is the normal way, but that is the way I would do it) 

     

  • (cs) in reply to John Bigboote
    John Bigboote:
    Anonymous:
    kuroshin:
    lock level 0 = unlocked

    lock level 1 = exclusively locked 

    lock level 2 = cannot be locked

    lock level 3 = superlock

    lock level 4 = minilock

    lock level 5 = record file not found

    lock level 6 = headlock

    lock level 7 = cockb-lock
     

     

    lock level 0.001 = null lock

    lock level 0.000001 = very null lock

  • thedoctor (unregistered) in reply to Tango Uniform

    Actually this quite a common problem in web applications.

    Timestamp is not safe because updates can be happening quite quickly (where batch processes are operating as well as user transactions). The most common solution is to add a version column to all tables and check this when doing an update. The hibernate documentation has a discussion about this issue.

     

     

Leave a comment on “I Think I'll Call Them &quot;Transactions&quot;”

Log In or post as a guest

Replying to comment #:

« Return to Article