• (cs) in reply to Grimoire

    I just put it in my sig.

  • jbode (unregistered) in reply to sammybaby

    I worked on a project where there were three distinct sets of tables to track whether a job was pending, executing, or finished.  When a job was pulled from the queue, the entries in the "pending" tables would be deleted and then written to the "executing" tables (yeah, in that order), and after the job was finished, the entries from the "executing" tables would be deleted and then written to the "finished" tables*. 

    This is approximately eleven orders of magnitude worse. 

    *To be fair, the guys who originally wrote the system were used to hierarchical-style database systems like Image on MPE, and didn't really grok relational systems.  Unfortunately, none of them saw that as a problem. 

  • (cs) in reply to SteveSenior
    Anonymous:

    We have better jobs for better programmers.


    Where? I find them impossible to find, generally... or they have impossible entrance requirements ie they only take people that have proven themselves competant... which I understand completely - you don't want to end up with another f'ing Gary/Jay... but how do new developers get a foothold into the right jobs?

    I have found so far that it's a slow crawl to get where you want to be. Unless you have the benefit of having been a geek mucking about with programming since you were ten (which I wasn't) that it's pretty hard to prove yourself capable.
  • (cs) in reply to JoeyLemur
    JoeyLemur:
    Imagine...

    I've seen this before, and I shudder to think that its happened elsewhere.


    sadly our system uses something similar. Not precisely a commerce system, but we do store transaction files for every transaction a user does on our website... we do store them in separate directories for each user... but several of our "users" are administrative - ie they handle multiple transactions for other customers (ie they are helpdesk doing transactions on the behalf of many of our users)... it's starting to take around a minute for their page to load these days... while they're on the phone to the irate customer. :P

    The app that displays the list of these transactions to the user is retarded in itself and has gotten to the stage where it's consuming some ridiculous amount of our server's CPU... to the detriment of the apps that actually do something useful (like allow our customers to actually buy stuff from us).

    Of course if I mention the fact that we should fix it I tend to get blank stares and comments about how much time that'd take (or "so it'll just take a couple of days then" ideas...

    Our sysadmin had begun to get onto it lately and we seem to be finally getting some way to convincing them it's necessary to put this stuff in the database... which is when I finally took a peek at our database code <shudder> I've submitted some of that to this forum already (aparrently it's hideous, but not uncommon enough to put up here or something).
  • (cs) in reply to me
    Anonymous:

    [...]
    This nonsense is very common. I have had to clean up several databases that had sets of tables (new set for each customer). A common reason given to me if I ask why this design was chosen was security. It is far too mind boggling to consider the rammifications of having all the data in one table (or set of tables).

    A variation on this is to use different tables for different time periods. Extra points for being used in conjunction with the above.
    [...]


    <font size="6">H</font>mmm.  Different tables for different time periods.  That sounds like it would be fun to create reports that span across two or more intervals.


  • (cs) in reply to Kooch

    My point about the study is that they're getting a self-selected subsample of the students at the top of the class, and similarly at the bottom.  This is a major problem in statistics; there's nothing you can do to fix this other than attempting to repeat the study on a true randomized sample.  If the self-selection doesn't depend on self-perception, then the study works fine.  I wonder if the APA had a statistician on their peer-review comittee?

    As for it being specific to college students, that's another point.  It's very hard to get a truly representative sample; just look at the major fights going on over the US Census and representative samples.  Still, it's worth the study, as the college student population may resemble others, such as programming.

    The study hits close to home for me; I'm a biostatistics grad student myself.  grin

  • Anonymous (unregistered) in reply to kipthegreat
    kipthegreat:
    welcor:

    kipthegreat:
    WTFer:

    /I'm wearing the juice, man!


    That sounds kind of hot

    Read the pdf above (it's on page one as well, so you don't have to spend too long to find that quote).

    *shakes head in disbelief* 
    If it wasn't in a scientific paper, I'm not sure I'd believe it. Actually it's so implausible (& incomprehensible & unbelievable) I'm not sure I believe it even if it IS in a scientific paper.



    I did read it.  Well the first seven pages anyway.  I guess you're one of those who would place themselves in the 68th percentile on your ability to recognize humor (meaning, you're really in the <25th percentile).


    And you'd probably rate yourself in the 54241231th percentile of funny people.
  • Steve Webster (unregistered) in reply to ammoQ

    Don't bet on it - I saw someone normalise gender out into its own table. His reasoning was that with the current advances in medical science, we'd probably have enough genders to warrant their own table in a couple of years. This was in 1999 and since stupid is an attribute of both male and female and not a separate gender, I guess he can't use himeslef as justification for the decision.

  • Andrey (unregistered)

    He'll be evaluating a new database system, you say?  I know just the right software for the next project.

    Let's try the sales pitch:
    "Comma Separated Values file (CSV) is a cutting edge technlogy that makes the current RDBMses obsolete!  CSV allows users to easily implement storage and retrieval of their data without the need for slow and bloated database systems.  The CSV system also provides robust facilities for cross-platform compatibility unmatched by the competitors!"

  • somebody (unregistered) in reply to xeno

    Ha, but would you really want to be promoted to whatever position he got? The position probably would have to be more pomp than skill with such unknowledgable management, and perhaps would of been the beginning of uncomfortable meetings and preparations for the scapegoat, if needed. :D

    On issue of the OP's post, I still can't believe this is real. Nobody would ever do this kind of work professionally. It's just too far fetched.

  • (cs) in reply to beliar
    beliar:

    Anonymous:
    Of course, the real WTF is that he's using Hungarian notation by starting all the names with 'tbl'.

    youre talking about "Systems Hungarian", right?


    I believe he was talking about ntnHungarian...
  • (cs) in reply to welcor
    welcor:

    kipthegreat:
    WTFer:

    /I'm wearing the juice, man!


    That sounds kind of hot

    Read the pdf above (it's on page one as well, so you don't have to spend too long to find that quote).

    *shakes head in disbelief* 
    If it wasn't in a scientific paper, I'm not sure I'd believe it. Actually it's so implausible (& incomprehensible & unbelievable) I'm not sure I believe it even if it IS in a scientific paper.


    I take it that you don't subscribe to This is True (http://thisistrue.com)
  • (cs) in reply to Andrey

    Anonymous:
    Let's try the sales pitch:
    "Comma Separated Values file (CSV) is a cutting edge technlogy that makes the current RDBMses obsolete!  CSV allows users to easily implement storage and retrieval of their data without the need for slow and bloated database systems.  The CSV system also provides robust facilities for cross-platform compatibility unmatched by the competitors!"

    Nah, you've got to store the data in XML and query it with XPath!  XML kicks CSV's @$$...[;)]

  • Enders (unregistered)

    I wonder if a manager came up and ask him can you give me a total of my sales / client or sales / product ?
    Or give me my top 10 best clients

    Well his responses could :
    The database wasn't designed that way because it was never a requirement
    Microsoft SQL server wasn't designed that way.

    I once saw a database where clients were deleted if they left. And because it was a relational database,
    their orders and order details were deleted too. You don't wanna have old orders in the database do you ?
    But hey what about the sales for this year ?



    And what is
    <font color="Red"> Something didn't quite work out ...
    - CAPTCHA Validation Incorrect </font>

  • mcguire (unregistered) in reply to Zair
    Zair:
    One problem I have with that study is that the participants were recruited from college students, with the promise of extra credit in a class.  So, who needs extra credit?  Those that think they could benefit from it: the middle of the distribution.

    The students at the bottom of the class (and knew it) wouldn't enter, because they know they'd fail regardless.
    The students at the top (and knew it) have no need of extra credit.

    So, they excluded these groups from their analysis, and it may have changed the results strikingly.  I'd like to see these studies carried out again with a true random sample from the students, not just those enticed by extra credit.

    As for the quote... I believe it.  It certainly requires more testing... maybe that guy just had a bad batch of juice?


    I don't know about Cornell, but at UTexas, psychology classes require participation in those kinds of studies.  There's a nominal payment, extra credit in this case I suppose, but it is a class requirement.  "Psychology" is the study of the psychology of psychology undergraduates....

    (No, I never took a psych class.  I just hung around UT for far too long.)

  • (cs) in reply to Zair

    Zair:
    My point about the study is that they're getting a self-selected subsample of the students at the top of the class, and similarly at the bottom.  This is a major problem in statistics; there's nothing you can do to fix this other than attempting to repeat the study on a true randomized sample.  If the self-selection doesn't depend on self-perception, then the study works fine.  I wonder if the APA had a statistician on their peer-review comittee?

    As for it being specific to college students, that's another point.  It's very hard to get a truly representative sample; just look at the major fights going on over the US Census and representative samples.  Still, it's worth the study, as the college student population may resemble others, such as programming.

    The study hits close to home for me; I'm a biostatistics grad student myself.  *grin*

    I can't get to the paper right now, but as I recall the key results of the study were not that most people think they're above average --that's been well-replicated in other studies, and one needs to look at all of them to see if sampling bias is could account for the apparent phenomenon. The key result of the paper is that there's a relation between the discrepancy in one's self-assessment of competence and one's actual competence, and also one's ability to learn to adjust one's self-assessment. Maybe I'm missing something, but I can't see how the sampling biases discussed here could account for these relations, except is some vague "not a random sample" way. The theory for the relation is rooted in general cognitive processes, which are common to all people, not just college students and not just college students that volunteer for studies. If you really want to criticize the study for this, you should lay out an alternative theory for the relation that implies that it only happens to such students.

    As for the hypothesized relation of between course performance and willingness to volunteer for studies, I don't recall there being any in my teaching experience, but my course records are all on 5.25 floppies and not readily accessible. Even if there is a relation, it seems unlikely to me that course performance would in turn be correlated with something like the ability to predict what's funny to other people. And even if it were such a relation, the "middle selection" scenario that Zair lays out would result in a restriction of range, whose effect would be to weaken the relation between one's discrepancy in one's self-assessment and one's actual competence. Restriction of range cannot create a relation that's not there.

    --RA

  • Ubergrunt (unregistered)

    Believe it or not, I've already seen too many projects like this in my life. All made by "excelent" Clipper programmers. By some incompreensible reason, the same guys are STILL programming in Clipper.

    Me? I'd put live 220V on the guy's keyboard and would wait for the scream. ;-P

  • Richard (unregistered) in reply to kipthegreat

    I can understand that incompetent people can be incompetent at recognizing their own incompetence, leading to arrogance. What I don't understand is how these people can often manage to convince so many other people that they are a "star". As I type this, a very prominent political leader comes to mind for some reason.

    Often because the people that employ them are less competent. It seems that IT is unusual in that the customers know so little about what they are buying - whether it's software, hardware or developer time.

    I dead hear a theory though that a certain political leader may actually be quite competent, but use the appearance of incompetence do disarm opponents.

  • (cs) in reply to Rank Amateur
    Rank Amateur:

    I can't get to the paper right now, but as I recall the key results of the study were not that most people think they're above average --that's been well-replicated in other studies, and one needs to look at all of them to see if sampling bias is could account for the apparent phenomenon. The key result of the paper is that there's a relation between the discrepancy in one's self-assessment of competence and one's actual competence, and also one's ability to learn to adjust one's self-assessment. Maybe I'm missing something, but I can't see how the sampling biases discussed here could account for these relations, except is some vague "not a random sample" way. The theory for the relation is rooted in general cognitive processes, which are common to all people, not just college students and not just college students that volunteer for studies. If you really want to criticize the study for this, you should lay out an alternative theory for the relation that implies that it only happens to such students.

    As for the hypothesized relation of between course performance and willingness to volunteer for studies, I don't recall there being any in my teaching experience, but my course records are all on 5.25 floppies and not readily accessible. Even if there is a relation, it seems unlikely to me that course performance would in turn be correlated with something like the ability to predict what's funny to other people. And even if it were such a relation, the "middle selection" scenario that Zair lays out would result in a restriction of range, whose effect would be to weaken the relation between one's discrepancy in one's self-assessment and one's actual competence. Restriction of range cannot create a relation that's not there.

    --RA



    Sampling biases can create nearly any relationship you can think of in statistics.  They're really the toughest problem that epidemiologists and biostaticians deal with; the good thing about the problems in the mathematics is that there's a right answer.  But how do we define what the "right" population is for a study?

    I anticipate that this isn't a fatally flawed study; like you said, it's probably nearly the entire class that participated in the study.  My point is that we have no way to verify that assumption.  I'm just laying out a what-if scenario for what could have gone wrong.  If there truly is no relationship in the population, it's still possible to detect a relationship in the sample if the sampling process is associated with the outcome.  (disclaimer: this is for prospective studies, not conditioning on the covariates that influence the association with the outcome.  This is exactly the scenario in the paper.)

    Also, it's really easy to create (or destroy) a relationship when restricting on range.  Take the association between daily water intake and death, for example.  If we restrict to the normal water intake amounts, it doesn't look like water can be unhealthy.  If we drop down to 0 liters per day, or up to 50 liters per day, then it's fatal.  Any relationship that isn't nicely linear can be affected by this, and it's very rare that we -know- a relationship is linear. 

    Alright.  I suppose I should stop posting off-topic so much.  *grin*  I think the paper is likely to be fine; I'm just saying that the association could have come about by biased sampling.  (unlikely, but very possible)
  • (cs) in reply to Zair
    Zair:

    Alright.  I suppose I should stop posting off-topic so much.  *grin* <snip>


    What?!?!? Off topic posting is the lifeblood of tdwtf! Unless, of course, you are not wearing your juice.

  • (cs) in reply to TankerJoe

    I wonder if you can spin juice into a fabric.

  • Chris (unregistered) in reply to diaphanein

    Lots of reasons...

    One company I worked for had a subsidaray. They had a "star." This guy couldn't code, I don't think he even understood the concept of programming. Once he turned off our products ENTIRE debug assert system because an assert was firing because of his code. LATER he turned off the memory managment error checks, cause he had memory leaks.

    He finally quit (we thought he was fired, but he actually got a better job offer from another company)

    After he left, we discovered that everyone at the other company LOVED him, and thought he was a star. They couldn't understand why everyone up here hated him with a passion.

    It turns out this guy was so prolific at writing code that he took off tasks from other peoples plates. He was prolific at wiriting code because he was good at, passing the buck to us, at hiding the bugs. Of course writing more code meant he wrote more bugy features.

    After he left, and the other company took over his code, they began to understand why we hated him.

  • AAA (anonymous admins of america) (unregistered) in reply to JoeyLemur
    JoeyLemur:
    Imagine a commerce system.
    Imagine that it doesn't use a database.
    Imagine that it writes each order to a seperate file.
    Imagine that it does this to the same directory.
    Imagine an ext2 filesystem in Linux trying to cope with 100,000+ files in a single directory...
    I've seen this before, and I shudder to think that its happened elsewhere.


    Imagine it's in a Stock Exchange;
    Imagine it's a COBOL program;
    Imagine the UNIX is Solaris 2.1 (ufs);
    Imagine that you have to use "find" because the sort function in "ls" quits with an out of memory error, I think that even the shell can't cope with that, as I got a lot of "arg list too long" errors;
    Imagine that it's used and keep beeing used because it's working...

    And then imagine that there is NO primadona, no star programmer... No, the development team did it, and it worked ok, so they were using it for years.
    Thanks God every year they start with an empty dir, but they generate about 100 files a day, not sure what the files had inside I think it was some kind of record with each order or query made by every user.

  • AAA (anonymous admins of america) (unregistered) in reply to b1xml2
    b1xml2:
    anyone in this day and age who is arrogant enough to re-invent the wheel ought to be publicly denounced.


    The problem is not when you re-invent the wheel. That's the way the code and algorithms evolve, if not everyone whould be using bouble sort.

  • jimbo (unregistered)
    Alex Papadimoulis:

    Steve received a frantic call from client management, begging him to help fix their largest client's order entry system. In production for less than a year, the system was running as fast as a crippled tortoise and, short of upgrading to Blue Gene, they had maxed out hardware options. But none of this surprised Steve; this was, after all, a jProject.

    jProjects, as they are known throughout the company, are projects destined to fail or turn into an unmaintainable mess. jProjects are, consequently, lead-up by the company's star developer, Jay. In addition to being wildly innovative, Jay manages to discover architectural flaws in programming, database, and other third-party tools when developing his own architecture.

    On the order-entry system, the slowdown was a result of Microsoft SQL Server improperly indexing table pages. Jay's design was brilliant: each order had its own table. This way, any new changes to the table template won't affect the old orders.

    [image]

    After spending a few days of combining the ungodly amount of order tables, Steve was able to increase performance by a few magnitudes and reduce the database size from 80GB to 2GB. Jay, still the star-developer, scoffed at the "hack" and said he will be evaluating different database systems for future projects.

  • bpb (unregistered)

    OMG, now I know that I have made some bad design decisions in the past ( I wish I could revisit them), but this is a disaster.

    What level of normalization is this?

Leave a comment on “Hacking the jProject”

Log In or post as a guest

Replying to comment #53279:

« Return to Article