• (cs) in reply to ¯\(°_o)/¯ I DUNNO LOL
    ¯\(°_o)/¯ I DUNNO LOL:
    Does anyone know where I can find a hooker who will give me an ETL job?
    Not sure about the T, but I imagine the EL is a pretty standard offering.
  • (cs)

    Offtopic: Are the forums down for anyone else?

  • My name indeed (unregistered) in reply to Nagesh
    Nagesh:
    If management want to save time and don't want to pay for license, I think NIgel is doing great job and I will also do same as Nigel. If code is 3000 lines is anyone doing code review before deploying this code? Is it truly returning 1 record per line of code? Are variables correctly and freely declared?

    All these questions are leaving NAgesh with many doubts in his mind. Please reply when you can. He is currently dealing with complex type problem in C#.

    I thought Nagesh was a Java programmer.

  • Jim the Tool (unregistered)

    Later, Nigel and Andy got together for a bit of the old in-out. Nigel was top, of course.

    Nigel exclaimed, after the first time, "it used to take thirty minutes, but now it's only ten!"

    Andy thought quietly, "there are only three thousand records, it should only take milliseconds," but did not complain.

    Captcha ludus. What's ludus is that so many complaints come in about the stories. Then again, I guess I could write a better story. Maor sex please.

  • (cs) in reply to n_slash_a
    n_slash_a:
    anonymous:
    I'm not entirely convinced that this isn't really a heavily anonymised story from inside the Obamacare website rollout.
    You think the Obamacare website can handle 3000 records at once?

    Do you think it even gets that many people signing up?

  • (cs) in reply to My name indeed
    My name indeed:
    Nagesh:
    If management want to save time and don't want to pay for license, I think NIgel is doing great job and I will also do same as Nigel. If code is 3000 lines is anyone doing code review before deploying this code? Is it truly returning 1 record per line of code? Are variables correctly and freely declared?

    All these questions are leaving NAgesh with many doubts in his mind. Please reply when you can. He is currently dealing with complex type problem in C#.

    I thought Nagesh was a Java programmer.

    He is, that's probably why he's confused with the simplest C# problems.

  • (cs) in reply to Lorne Kates
    Lorne Kates:
    Offtopic: Are the forums down for anyone else?

    Always a good sign when a site author asks this question.

  • (cs) in reply to Lorne Kates
    Lorne Kates:
    Offtopic: Are the forums down for anyone else?
    Yes.
  • Peyote Short (unregistered)

    Sometimes there's bits of a story that I don't understand because I've never worked with big corporate data. This particular tale is entirely opaque to me. I guess the gist of it is that the consultant is writing code that processes records inefficiently.

  • (cs) in reply to Shoreline
    Shoreline:
    robbak:
    OK, with a whooshing sound over my head, I proceed to explain the joke.

    Consulting firm produces system. It takes 30 minutes to complete.

    Andy grabs a few evaluation licenses and puts together a functional version that handles a few thousand records in milliseconds.

    Nigel the Expert sniffs and walks out. He reworks the system so it takes only 10 minutes.

    Andy looks at how many records the system is handling. It's only a few thousand.

    So Andy's afternoon produced a number of systems that do in milliseconds what and entire consulting firm's do in 30 minutes. and Nigel the Expert's weeks-long effort did in 10 minutes.

    I got the impression it was also implied that 3000 javascript lines were printed to handle specific records.

    There's a reason the story focused on Nigel being heralded as a genius for making the process marginally faster with a small workload. It was to show that Andy's solutions would have been much faster since his test workloads were essentially the same as what Nigel's process was handling.

  • (cs) in reply to Peyote Short
    Peyote Short:
    Sometimes there's bits of a story that I don't understand because I've never worked with big corporate data. This particular tale is entirely opaque to me. I guess the gist of it is that the consultant is writing code that processes records inefficiently.

    The moral of the story is that rockstar programmers aren't necessarily better than off-the-shelf solutions.

  • J (unregistered) in reply to ratchet freak
    ratchet freak:
    Zylon:
    ratchet freak:
    dtech:
    What exactly is a "nosebleed-level executive"? Google turns this article up as the only place it's used...

    it's a play on nosebleed section:

    wikipedia:
    The nosebleed section (or nosebleed seats) are those seats of a public arena, usually an athletic stadium or gymnasium, that are highest and, usually, farthest from the desired activity.
    Uh, no. Think WHY the nosebleed section is called that. It's because you're sitting at a high altitude. So "nosebleed-level executive" would denote a very high-level executive.
    They are still far away from the action so it still fits :P

    So, TRWTF was using a sports analogy on this website?

  • (cs) in reply to J
    J:
    ratchet freak:
    Zylon:
    ratchet freak:
    dtech:
    What exactly is a "nosebleed-level executive"? Google turns this article up as the only place it's used...

    it's a play on nosebleed section:

    wikipedia:
    The nosebleed section (or nosebleed seats) are those seats of a public arena, usually an athletic stadium or gymnasium, that are highest and, usually, farthest from the desired activity.
    Uh, no. Think WHY the nosebleed section is called that. It's because you're sitting at a high altitude. So "nosebleed-level executive" would denote a very high-level executive.
    They are still far away from the action so it still fits :P

    So, TRWTF was using a sports analogy on this website?

    It's also a fairly common reference in regards to opera houses

  • eric76 (unregistered)

    Does anyone else get the feeling that all he did was change the code to sleep 2 seconds between records instead of 6 seconds?

  • eric76 (unregistered) in reply to eric76
    eric76:
    Does anyone else get the feeling that all he did was change the code to sleep 2 seconds between records instead of 6 seconds?

    Oops. That should have been 0.2 seconds and 0.6 seconds, respectively.

    That would make it 300 records a minute or 3000 records in ten minutes instead of 100 records a minute or 3000 records in thirty minutes.

  • monkeyPushButton (unregistered) in reply to nonpartisan
    nonpartisan:
    Lorne Kates:
    Offtopic: Are the forums down for anyone else?
    Yes.

    Only the Sidebar page. Individual threads can be seen and all other topics, but I get an error on going to the Sidebar topic page.

  • Javelin (unregistered) in reply to eric76
    eric76:
    Does anyone else get the feeling that all he did was change the code to sleep 2 seconds between records instead of 6 seconds?
    Nah. More like 200ms instead of 600ms.
  • sqlblindman (unregistered) in reply to tin
    tin:
    What happened after Andy found out there was only 3000 records being processed? What was causing the slowdown?

    He hired a real DBA to write sql script to process the records using set-based transactions. And the process now completes in under 2 seconds.

    And they all lived happily ever after.

  • Manfred the Manly (unregistered)

    Yeah TRWTF is that anyone can think that 10 minutes to handle 3000 records in a modern DB setting is 'acceptable'.

    If my 3000 record query doesn't return in 2 seconds I start kicking over contractor cubicles!

    ALL SHALL FEAR MANFRED THE MANLY!!! RAAAWWWWRGGG

    captcha: tristique - when you're in a tryst but don't know who's on the other end...

  • anonanon (unregistered) in reply to There was a punch line

    Brilliant!

  • (cs) in reply to Zylon
    Zylon:
    ratchet freak:
    dtech:
    What exactly is a "nosebleed-level executive"? Google turns this article up as the only place it's used...

    it's a play on nosebleed section:

    wikipedia:
    The nosebleed section (or nosebleed seats) are those seats of a public arena, usually an athletic stadium or gymnasium, that are highest and, usually, farthest from the desired activity.
    Uh, no. Think WHY the nosebleed section is called that. It's because you're sitting at a high altitude. So "nosebleed-level executive" would denote a very high-level executive.

    You know I looked around, for faces I'd know, I fell in love with the people in the front row

  • Atle (unregistered) in reply to Manfred the Manly

    It was reduced from 30 min, and there were 3000 lines of code taking a month to develop, so obviously there is some complex processing involved as well, and not just simple data shoveling.

  • Nigel (unregistered)

    Obviously Andy is clueless of the actual problem that is being worked on. The story contains no evidence at all

    He's playing with an evaluation copy of ETL software for 3000 records?? WTF a simple query would do in that case.

  • Bigus Dickus (unregistered) in reply to Martin

    [quote user="Martin"]Frist, they moan about the dumbing down. Then they moan when left to draw their own conclusions. I think the sensible message to take away from this is that people will moan.[quote]

    Dunno about that. You are mother is more of a squealer.

  • Fritz, a.k.a. Fritzo (unregistered) in reply to Manfred the Manly

    [quote user="Manfred the Manly"]Yeah TRWTF is that anyone can think that 10 minutes to handle 3000 records in a modern DB setting is 'acceptable'.[quote]

    Still a lot faster than anything in SAP.

  • Hans (unregistered) in reply to Moo
    Moo:
    The capital of Germany is Berlin.
    So's the capitol.
  • John Max (unregistered)

    Andy can talk whatever he wants, but i do believe action speaks louder than words.

    at least the stupid nigel improve something. what did Andy do? just watch and complaint?

  • WowImpressed (unregistered) in reply to ih8u
    ih8u:
    WowImpressed:
    Rather drop TheDailyWTF to a weekly TheWeeklyWTF and up the quality of the stories. And no, I'm not going to send in my own again; they never get published.

    I cannot imagine, then, how terrible your stories must be.

    Good point.

  • mv (unregistered) in reply to tin

    3000 records in 10 minutes is still god-awful-by-orders-of-magnitude horrible. The last time I worked on an application where I had any real control over the I/O subsystem, I got it to where it could write around 40,000 records per second (and read over 300,000 records per second).

  • caper (unregistered)

    which he built directly into the web code,

    I don't understand what that means in the context of ETL.

  • (cs) in reply to John Max
    John Max:
    ... what did Andy do? just watch and complaint?

    Apparently you don't know what "evaluating" and "test run" mean. I'd recommend finding out. They're kind of useful concepts.

  • Tasty (unregistered) in reply to Nagesh
    Nagesh:
    If management want to save time and don't want to pay for license, I think NIgel is doing great job and I will also do same as Nigel. If code is 3000 lines is anyone doing code review before deploying this code? Is it truly returning 1 record per line of code? Are variables correctly and freely declared?

    All these questions are leaving NAgesh with many doubts in his mind. Please reply when you can. He is currently dealing with complex type problem in C#.

    Are complex types supported in C#? Time for FORTRAN!

  • (cs) in reply to Hans
    Hans:
    Moo:
    The capital of Germany is Berlin.
    So's the capitol.
    notsureifserious.jpg, so I'll assume you are serious.

    Nope. "Capitol" only refers to a building. A city that is the seat of government for some state is a "capital," just like Moo said.

  • Norman Diamond (unregistered) in reply to dtech
    dtech:
    What exactly is a "nosebleed-level executive"? Google turns this article up as the only place it's used...
    http://www.youtube.com/watch?v=U4Dw4sux8FY
    
            
    
        
    • 
        
  • 
 Norman Diamond   (unregistered)                                         in reply to Lorne Kates
            
            
    
                
     Lorne Kates:
    Offtopic: Are the forums down for anyone else?
    Yes, I can't see your question.
            
    
        
    • 
        
  • 
 Norman Diamond   (unregistered)                                         in reply to Tasty
            
            
    
                
     Tasty:
     Nagesh:
    Please reply when you can. He is currently dealing with complex type problem in C#.
    Are complex types supported in C#? Time for FORTRAN!
    GOD is real, unless declared complex.
            
    
        
    • 
        
  • 
 JRI   (unregistered)                                         in reply to Moo
            
            
    
                
    The capital of Germany is "G"
    
            
    
        
    • 
        
  • 
    (cs)                                         in reply to Norman Diamond
            
            
    
                
     Norman Diamond:
     Lorne Kates:
    Offtopic: Are the forums down for anyone else?
    Yes, I can't see your question.
    
The daily WTF forums (the latest topics of which you see in "Side Bar WTF") are a completely different software than the article comments.

    For example a good alternative toepic-WTF-variant of BB-Code works in the article comments, but does not at all work in the comment forums.
    

    Use two totally different techniques for exactly the same task is the level of WTF you expect on a page with such a name.
    

    It delivers on the promise! It's WTFs all the way down...
    
            
    
        
    • 
        
  • 
    (cs)                                         in reply to JRI
            
            
    
                
     JRI:
    The capital of Germany is "G"
    
Nice spot!
            
    
        
    • 
        
  • 
 Carrie   (unregistered)                                         in reply to Steve The Cynic
            
            
    
                
     Imsoconfused:
    like anyone is humble when they say imho
    

    Doesn't that mean 'In My Honest Opinion'?
    
            
    
        
    • 
        
  • 
 Ema Nymton   (unregistered)                                     
            
    
                
    Where's the rest of the story?
    
            
    
        
    • 
        
  • 
    (cs)                                         in reply to tin
            
            
    
                
     tin:
    I'm not normally one to wonder where the punchline is, but is this part 1 of 2?

    What happened after Andy found out there was only 3000 records being processed? What was causing the slowdown?
    

    We need answers, people!
    
because it was the presidents daughter
    
            
    
        
    • 
        
  • 
    (cs)                                         in reply to Atle
            
            
    
                
     Atle:
    It was reduced from 30 min, and there were 3000 lines of code taking a month to develop, so obviously there is some complex processing involved as well, and not just simple data shoveling.

    
No 2000 of the lines was a new Date/Time class
3 were something about a 
double cheeseburger;
char broiled;
long time;

    500 were blank lines
200 were comments
    

    295 lines for an easter egg
5 lines for actual fix
    
            
    
        
    • 
        
  • 
 Norman Diamond   (unregistered)                                         in reply to accident
            
            
    
                
     accident:
    No 2000 of the lines was a new Date/Time class
    So THAT's where the Y2K problem came from.
            
    
        
    • 
        
  • 
 tl   (unregistered)                                         in reply to Imsoconfused
            
            
    
                
     Imsoconfused:
    What the heck does ETL stand for? I hate it when people use acronyms without defining them in the article, I couldn't even find one appropriate on acronymfinder.com:

    Early Termination Liability?
Eat-To-Live Diet?
Echo Train Length?
Economic Threshold Level?
Edge-Triggered Latch?
Edison Testing Laboratories?
Educational Technology Learning?
Effective Translational Lift?
El Toro Loco?
Electric Testing Laboratory?
    

    IMAO (In My Arrogant Opinion (like anyone is humble when they say imho...)), If it's not in the first 10 items on acronymfinder.com it doesn't exist!
    

    Did you know acronymfinder.com shows definitions in alphabetical order?
    
            
    
        
    • 
        
  • 
 Manfred the Manchilder   (unregistered)                                     
            
    
                
    I always thought imho means in my honest opinion...
    

    captcha: Facilisi - An italian facist OR a facist khalasar's wife
    
            
    
        
    • 
        
  • 
 anonymous   (unregistered)                                         in reply to Manfred the Manchilder
            
            
    
                
     Carrie:
     Imsoconfused:
    like anyone is humble when they say imho
    

    Doesn't that mean 'In My Honest Opinion'?
     Manfred the Manchilder:
    I always thought imho means in my honest opinion...
    The "h" can stand for either "humble" or "honest", but honestly, why would you honestly feel it necessary to tell people honestly that you're honestly sharing your honest opinion? Then again, "humble" isn't much better.
    
            
    
        
    • 
        
  • 
 Rhett Torek   (unregistered)                                         in reply to anonymous
            
            
    
                
     anonymous:
    The "h" can stand for either "humble" or "honest", but {color=inherit;display:none}honestly, {/color}why would you {color=inherit;display:none}honestly {/color}feel it necessary to tell people {color=inherit;display:none}honestly {/color}that you're {color=inherit;display:none}honestly {/color}sharing your honest opinion?{color=inherit;display:none} Then again, "humble" isn't much better.{/color}
    On the internet, without {color=inherit;display:none}or with {/color}emoticons or haemoglobins, it's hard to tell if a statement is sarcastic or not.  If they don't say their opinion is honest or humble, readers might assume they're rhetoric.

    [/humbug]
    
            
    
        
    • 
        
  • 
 anoymous coward   (unregistered)                                         in reply to Jeff
            
            
    
                
     Jeff:
    ETL - Export, Transform, Load

    basically a process to take data out of one database, massage it, and put into another database.
    

    I took it to mean "ab initio".
    
            
    
        
    • 
        
  • 
    (cs)                                         in reply to Rhett Torek
            
            
    
                
     Rhett Torek:
     anonymous:
    The "h" can stand for either "humble" or "honest", but {color=inherit;display:none}honestly, {/color}why would you {color=inherit;display:none}honestly {/color}feel it necessary to tell people {color=inherit;display:none}honestly {/color}that you're {color=inherit;display:none}honestly {/color}sharing your honest opinion?{color=inherit;display:none} Then again, "humble" isn't much better.{/color}
    On the internet, without {color=inherit;display:none}or with {/color}emoticons or haemoglobins, it's hard to tell if a statement is sarcastic or not.  If they don't say their opinion is honest or humble, readers might assume they're rhetoric.

    [/humbug]
    

    Hence my green comments that I've been posting recently.
    
            
    
        
    • 
    


            


            
  • 



            


                

                
Leave a comment on “Record Setting Performance”


                
Log In or post as a guest


                
                
Replying to comment #:


                

                    
                    Log Out
                


                

                    
                


                





                

                    
                


            


            
« Return to Article