- Feature Articles
- CodeSOD
- Error'd
-
Forums
-
Other Articles
- Random Article
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
Ok... I know you must be joking, but I feel the need to call out your O() analysis here, because it reflects a mistake many people seem to make in real analysis.
In this situation, N is the number of customers. (That's how you got to O(log n) for a single table, yes?) So, to call the complexity O(1), you must claim that the operation will never slow down no matter how many customers you have. For the WTF solution, that means you're claiming that the DBMS won't slow down no matter how many tables you create. This isn't something you normally would worry about with a sane design, because the number of tables would normally remain insignificantly small in this regard; but for O() analysis you account for growth without bound, and in this case that means an arbitrarily large number of tables . Eventually, whatever scheme your DBMS uses to find the table's data will either (a) prove to be worse than O(1) wrt number of tables, or (b) crap out entirely. Why? Because cool though they are, DBMS's are not magic.
The error I want to call out is: only looking at part of the operation in question -- the part that's obvious to the analyst, I suppose. In this case, you're only evaluating the cost of the table scan, and because the table size is fixed wrt the number of customers, that step is O(1); but that's only part of the process.
EOR
Admin
I think I screwed up the curve when I went for my diploma. In one class, I had 97%. The next nearest had 78%. On one assignment, facing student protest because of unclear instructions, the instructor bumped everyone up one mark (10%), except for me: I already had 10/10.
Sincerely,
Gene Wirchenko
Admin
He has one. Maybe several. How it's implemented depends on the particular DBMS being used, and the original designer probably doesn't even know about it, but it's there, and it probably gets scanned during query compilation (which, under this inane design, probably means "once for every query execution").
I am speaking, of course, of the table data in the system catalog (or dictionary, depending on your religion).
Admin
Row (well, customer) level security.
OK, this still isn't the right solution to that problem, but that is a problem that it solves.
Admin
I still have nightmares (no, really) about my Fluid Mechanics exams where the grades were between 1 and 5 (out of ten) and the 2 people with the 5s passed.
Admin
We do this, sort of... instead of having a 'hotel' column in our tables, we have a separate database for each hotel.
In our case, the justification is that each hotel is anywhere from 5-25 GB, and having a single database of many hundreds of GB would be kinda unweildy - especially since we have to move a hotel from one database to another from time to time.
So it CAN be a good idea... or at least less bad than the alternatives.
Admin
I feel the need to call out your call out.
I am well aware of how to do proper complexity analysis.
Way to ruin my joke, jackass.
Admin
Same at Victoria University in New Zealand.
Admin
Third year and required at Thompson Rivers University in Kamloops, British Columbia, Canada. You can get credit for it if you got at least a B+ in the databases course in the diploma program that I took (since the material is substantially the same). There is also a fourth-year database course. That one is not required.
Sincerely,
Gene Wirchenko
Admin
Ha! I took that class. The thing I remember most was trying to work on the crap server they'd set aside for my 300 person class...Oh, that and the prof was obsessed with mSQL. Not MYSQL, mSql. I ended up finishing my database on my desktop in mySQL, and jsut exporting the whole thing as a bigass SQL query, because the server was so bogged down with people executing inefficient sql. I think he wanted to grade me down, but didn't quite dare.
Decent class though. Like a lot of stuff at Rutgers with was hard as hell...I'm not sure what this says about the program, because you're right, the curve covered a lot of ground, so if the stuff was hard enough, it was a very fine line between the competent and the incompetent, and you know some of the latter ended up grouped with the former. Exam was a bitch, iirc...some of those queries were more complex than anything I've ever needed in real life.
Admin
why does everyone get so excited about database blunders? there's plenty of highly skilled programmers who don't have a clue about databases. as a games programmer, i have zero need to ever learn the pain of db design, so why bother? looks kind of boring to me!
Admin
Consequences.
Unfortunately.
As long as you understand that. If you move into other application areas, be prepared to have to learn. (It might benefit what you do now, but I will leave the choice to you.)
Among other things, sometimes people can die as a result. In comp.databases, I asked Joe Celko about a claim he made. The abbreviated bit is part of my question. The rest is his:
>> .. case you have mentioned about medicine and not knowing normalisation and a possible resultant loss of life? <<
A charity that ships medical supplies to Africa has a volunteer do a
database for them. They can get the supplies broekn down into smaller
than normal units for shipping. So instead of a minimum shipment of a
25 unit box of antibotics that a regular coimmercial buyer would have
to order, the supplier will break them out into the smaller 5-unit
boxes and some extra packing to see that they do not break or spoil on
the way to a war zone. This is a lot of work for the suppliers, but it
is a charity so they eat the extra cost, insane deadlines and
bullet-proof wraping paper required.
The kid that did the database put all of the available quantities in
one column as a comma separated list, like ('05,10,25;50') in the
database. His front end then displayed that same list and used a
substring to fill in the order quantity as a string. So we have a 1NF
violation and the wrong datatype.
The charity now wants to get the suppliers to break things down even
further to single units, so they can make up small field emergency kits
in the US. These kits can be distributed faster. It is more likely
that some of the kits will survive transit and get to their
destinations, so a lose of one order in transit will not be as bad.
The quantity column in the database was changed to ('01,05,10,25;50')
but the front end stays the same. SUBSTRING (qty, 1, 2) is now '01' in
the back end and '05' in the display in the front end. The result was
a clinic full of sick kids, many units short of critical medications
and not able to run down to the corner drug store to re-fill the
medicine cabinet.
Another example of a bad medical database was in an Atlanta hospital.
It would drop orders. The first month or so of operation the
administration loved it. You can save a lot of money billing people
for drugs and treatments they do not get. Medical personnel caught
this one pretty fast and the system was dropped at the cost of a few
million dollars.
I did not get called in on it, so I have no idea what the problem was.
My wife worked there for 13 years.
Sincerely,
Gene Wirchenko
Admin
My scores were so high ....
oh, actually it was me that was so high
Admin
Hmmm .. what's kind of things are stored in database? Bank accounts? Medical records? Personal Information?
Hmmm .. what kind of things happen in programming without databases? Games? Info-only web sites?
Admin
How would the professor know that he had created a test that an average student should score 75% on? He'd have to give the test to hundreds of people of known skill level before giving it to the class. Curving is a far more useful solution.
Admin
Ahhh..... that must be because most modern database system can't handle tables larger than 100GB. Oh, wait.... Oracle, MSSQL, and DB2 can easily handle tables far larger than that, as long as the indexing is OK and the queries are written by competent people.
Actually, just today, I was working on a table that was about 1GB. A query was taking 38 seconds to return 2 rows out of 6 million. A few tweaks to the query (none to the indexes) and it was down to 1 millisecond. Someone could easily have made the conclusion that the database server "couldn't handle 6 million rows" if they hadn't had a professional look at the situation.
BTW, how can you occasionally need to move a hotel from one database to another if there is a database for each hotel?
Admin
holy mother function..........
Admin
-3NF?
Admin
"cubicle mate"
That in itself is a WTF, as if cubicles are not bad enough when you are the only person in one.
Admin
Oh yeah. Someone ought to write some "Bastard DBA From Hell" stories. =)
Admin
Well hello there.. Actually games imply some database, but game programming in itself has little tangency to it. Here are a few more examples where the dev can do well without db proficiency:
- Industrial / embedded systems programming(like the ESP from your car, or auto-pilot on airliners)
- Entertainment applications - audio/video decoding, playback
- Scientific calculus / simulation software.
As for games, sure they are not vital, but they are good to have around. They are also a tough field, and although we have plenty of wtfs here as well, they are far more noble than presentation websites. We're few, we're proud and we... uh never mind. Here's some code for your joy:
if(rand()%5)
updateFrame()
enum(0,1,2,3,4,5,6,7,8,9);
void InitUI()
{
arrBtns[0].SetPos(231,234);
arrBtns[1].SetPos(231,244);
arrBtns[2].SetPos(231,254);
arrBtns[3].SetPos(231,264);
arrBtns[4].SetPos(231,274);
arrBtns[5].SetPos(231,284);
arrBtns[6].SetPos(231,294);
arrBtns[7].SetPos(231,304);
//...
}
Devildog
Admin
(turn on your sarcasm detectors)
sure there is. It completely avoids table lock blocking problems when you try to update 2 customers at the same time :). Indexes are so passe
btw:
This is a temporary hack until I can get the real CAPTCHA validation working. Just type broken in the textbox below.
<!-- HIP:ImageHipChallenge Run~at="server" Height="50px" Width="250px" id="wtf_HipImage" / --><!-- p style="margin:3px;">Prove that you're not a robot. In the textbox below, type in the word you see in the above image.</p --> <!-- HIP:HipValidator ID="wtf_HipValidator" Run~at="server" Text="* That's not it" CssClass="validationWarning" ControlToValidate="wtf_HipText" HipChallenge="wtf_HipImage" ErrorMessage="- CAPTCHA Validation Failed" / -->Note that registered users don't have to do this.
I am registered but can't login
Admin
True for embedded and low-level stuff, but you might be surprised how common databases are in automation systems.
So how many programmers are actually working on codecs? A few hundreds worldwide, I'd guess.
I'm not an expert in that domain, but I think many simulations require databases as well.
Admin
ammoq: Sorry about the codec example, just trying to fill the list :-|
True, databases are in all these fields, as well as driver/OS programming, just came to my mind. But that's because they handle data, right? Anything that handles a lot of data requires databases, in fact we have our own db guy at our small gd office. But it can be handled separately from the core systems of the applications I just enumerated. I don't need to know rDB design or any kind of sql to develop a game component, or an automation system component. Give the DB what belongs to the DB.
I gather Bus Raker was using the bank account / medical data argument to underline the importance of database programming. But it's still not interesting or challenging from an engineering standpoint, because the solutions are few in number and low in diversity, the main difficulty is in learning the technology wich is a mess, the rest just comes with experience. Yet it is clear, from everyday user experience, as well as from the countless examples of fine code posted on this website, that the care for writing good code does not grow with the importance of the handled data itself.
And it does not have to, as well as the clients accept the end result. In the fields I enumerated at the beginning, it's harder to sell a badly written product, it just would not be accepted by the buyers.
Devildog
Admin
This is a common misconception among youngsters. To put it right: I'm not a database programmer. I'm a solution developer who uses a lot of tools, including databases. My job includes really amazing projects, e.g. in the domain of logistics automation, and while it might be funny for a game developer to see a robot army walking on the screen, it's IMO much more intriguing to see real robots (i.e. laser guided vehicles, rack feeders etc.) moving around in a real big storage facility. I couldn't do the job without profound knowledge in how to use a database, but the focus is on the application domain, not the database.
Admin
At least he had the courtesy to keep them all in a single database. He could've created a new database for each new customer and we can only wonder why he didn't.
It sure is a little funny, just not haha-funny but retard-funny. You get the same feeling when you're laughing at a handicapped person who falls down from the wheelchair and can't get up.
Admin
ammoQ:
Don't know how, but you guessed right about my age group, and it's true I never touched a system on the scale that you describe there, but the solution itself, as you develop it, can be broken down into smaller, less scary problem. And I don't think the data management rises too much above routine. I worked close to a team trying to develop an MMORPG(unfortunately the project died), and the DB programming was the thing that would take the least development effort.
Would you say that the DB system which handles your logistics automation system data is about on the same level of 'intriguingness', or as interesting to develop as programming an autonomous robot's routines?
It's true I don't have much to do with databases and I'm out of my field here, but from what I do know, I'm sorry to say that I find DB most uninteresting. Probably my view would be different if I had to work with solutions on that scale, than I guess it's my loss.
Devildog
Admin
That was easy. I thought like you when I was in your age ;-)
I look at it like this: the autonomous robot's routines do the micromangement, my system does the macromanagement. Is it more intriguing to tell each motor when to start and when to stop, or to orchestrate the whole automation chain so it is able to process 10000 orders per day?
I don't think there is a general answer to that question.
Admin
Another fine example of this incredible technique, just discovered in a german hardware store:
http://www.kmelektronik.de/main_site/main.php?action=Product_Detail&ArtNr=abc&Shop=xyz
(
Word explanations for the german-unsavy:
Admin
The required knowladge to learn how to do this properly is in just about every A-Level Computing and ICT course I can find the details of.
Admin
About databases:
Databases are ugly and a hack because erros are runtime. On a C type checking is compiled time. So some errors are imposible. You can't even try to store a string data on a float. The compiler will stop you with a lovable error, and show warnings all around. This Is A God Thing.
But a database acept this on compile time:
query = "SELECT BuildDate FROM builds WHERE BuildDate='32'";
..this will compile, but is a error. The compiler will not stop you.
A fix to block this hole is Stored Procedures. Imho,.. Stored Procedures Are A Good Thing.
Other than that (runtime error evilness) database are a must on most if not all applications.
I also code games, so often I see databases as slow and ulgy and boring stuff . But most people work on Slow and Boring and Ugly stuff, so a database is the right thing. 2NF databases, not strange byzarro crap like this one.
--Tei
postdata:
This also compile and is a error:
query = "SELECT BuildDate FROM builds WHERE BuildDate='31-31-2005'";
query = "SELECT BuildDate FROM builds WHERE BuildDate='2005-1-31'";
..
using databases slip errors to runtime. And runtime erros are the most EXPENSIVE ONES!
Admin
How the hell is this a justification? Major players of the DB field (Oracle, SQL Server, DB2) have no trouble handling ungodly amounts of data (in terabytes, that is). Plus I don't see how you can move hotels between DBs since you have a DB per hotel, I don't see the point of moving hotels between DBs if you have a sane design (and if you ever need to you can anyway), and you create unnecessary complexity to retrieve data spanning the whole system (statistical data over the complete hotel chain for example).
The issue is that this kind of schemes is stupid, in France it led to the complete depreciation of the Baccalaureat (the national test we pass at the end of high school basically): it used to be a tough test, then it was decided that 80% of the french students trying it should have it. Except that they didn't put more money into the education itself, they lowered the barrier of entry, and kept lowering it.
And the Baccalaureat isn't worth shit anymore, and high school students don't get exposed half of what their parents were supposed to learn because the educational part was dumbed down as well (in order to reach the 80% instead of actually educating the students).
A test should be a bench of the students against requirements, it has an absolute value, not a relative one, if you're creating a test so that the average student should score 75% then the test is a failure, you could just weed out the bottom of the class without a test and be done, no test passing and no marking for strictly the same result.
Admin
Admin
Welcome to the wonderful world of preprocessors. For example, Oracle's preprocessor checks all spelling, the table names, the column names, privileges etc. And it saves a lot of work, too.
Admin
This one looks like having a copy of each table per shop. Though it would be preferable to have one database instance or at least one scheme per shop, there might be restrictions (coming from the web hoster) that force them to do it like that.
Admin
Yeah, except that C's type system is so weak and full of holes that
Had you at least talked about languages such as Ada or Haskell I may have agreed, but using C to criticize DB's runtime errors is a fucking joke.
Admin
Above site is also prone to SQL injection since it escapes ' and ", but not `
Check:
http://www.kmelektronik.de/main_site/main.php?action=Product_Detail&ArtNr=abc&Shop=x`yz
Admin
All C requires you to do is put a cast in your code. It doesn't verify that it won't turn into an error -- or even worse, a memory leak if you cast a larger pointer type to a smaller one.
Nobody writes SQL like you have above if they expect any quality at all is their code. That code deserves all the run-time errors it gets (not to mention SQL injection vulnerabilities). If you code it like this you'll never get a runtime error (This is VB.Net, but every language worth a crap has a way to do this):
<FONT face="Courier New">Dim buildDate as DateTime = dtpickerBuildDate.Value
Dim cmd As New SqlCommand("SELECT BuildDate FROM Builds WHERE BuildDate=@BuildDate")
cmd.Parameters.Add(New SqlParameter("@BuildDate", buildDate))
dr = cmd.ExecuteReader()</FONT>
Of course I've got three tiers of code smushed together is the example, but that's beside the point.
Stored procedures don't solve the problem either. In MS SQL, if you spell a table name wrong then it figures that maybe the table doesn't exist yet and lets you get away with it. Oracle procedures are even more fun. Sometimes procedures that access remote objects will break the first time you call them, and then fix themselves the second time. Besides, you'll still get the runtime error on the call to the stored procedure.
As for slow and boring and ugly, that's perspective. I'll bet that the original guys at eBay weren't very bored during the late 1990's. And I see that the game programmers over at EA had to file a class action lawsuit against their own employers to get working conditions to be tolerable.
Admin
Unintentionally funny?
This looks suspiciously like a MySQL-ism. Unless, of course, the BuildDate column is set as a VARCHAR or something. On my PostgreSQL systems, on the other hand:
<font face="Courier New" size="2">ERROR: date/time field value out of range: "32"
HINT: Perhaps you need a different "datestyle" setting.</font>
I think they are a God thing, also ;).
Yep. Confirmed. It is MySQL.
So, the real problem you are complaining about is not databases in general, but a particularly bad implementation of the relational model (actually SQL by definition is not completely relational). The whole point of the relational model, as expressed by Codd and other originators, was specifically to prevent bad logic, by providing the most direct way to turn a formal specification for your data into machine-manageable form. This logical model included complete type-checking for every query, and in fact a MUCH more rigorous approach to typing that you will find in C or many other languages.
As for developers thinking databases are boring and ugly, that is mainly due to lack of education and imagination. Well, and the poor implementations by most database vendors. Much more is possible, but instead we now get horrible hybrid XML/OO/SQL monsters like the new DB2, and who knows what trend is coming round the corner.
Admin
Heh, on second thought, I guess your post could mean that your programming environment doesn't have a way to check for compilation of strings to be passed to an SQL database. Umm... that's even more of a WTF, unfortunately. At the level you are talking about, it is of course just a string, so it has absolutely nothing to do with the database itself yet. But anyone who does serious database programming does not just embed strings of SQL into their code. There are libraries for that. Also there is such as thing as the prepared query. Also, there is such a thing as input validation. And views, user-defined functions, even user-defined datatypes. Plenty of tools to help you get that database interaction right.
Admin
I think you are wrong with that "memory leak". free() doesn't care about the pointer type. The pointer is casted to void* anyway.
Admin
At University of Turku (Finland) Databases is a mandatory course for all, and of course it is not about "how to use" a relational database but about how to design one. And on the information processing line (as opposed to the systems line, which deals with human-computer systems) many people end up taking also optional Advanced Databases, which is about how a RDB actually works internally.
Admin
..."but it actually works pretty well."
LOL -- I'm sure it does, I'm sure it does...
Admin
For what I've been talking about, it's not true that I don't have a clue. It's just not my speciality, but I do have a clue about the principles of database design and programming, and not from this site. I read this site for entertainment, mainly.
The way you feel so easily insulted by a side debate over general principles and matters of taste or inclination is not exactly unique on this forum, but still bothersome. It is not the way a debate should be carried. ammoQ for instance provided me with counter-arguments and a different angle on the whole issue. Your argument is just old-lady style word twisting. Well done.
Devildog
Admin
I agree.
Admin
Admin
Funny, I somehow have believed that things like "data model" along with design guidelines are supposed to be documented in architecture desription or similar document. Perhaps this should have been the first place to look for information.
/me still believes in crap like documentation....
Admin
And another problem with that would be the 2k character cap when executing a dynamic query
Admin
Hmm, from my point of view, this one is... interesting.
You see, I work with a non-relational db that is, in some ways, structured a little like this. The main difference being that our db is designed from the base up to work like this - not as a really nasty unscaleable hack.
As to "why", well the bean counters may want to do nothing but aggregates on tables, but in a clinical environment (where I work), a doctor generally just wants to see the data that relates to the patient sitting in front of him - and the server is slow, the network is two tin cans and a piece of string....
That doesn't stop this WTF being truly 'orrible...
Admin
Sorry, doesn't work. I majored in English, and I still know more about designing relational databases than most people.
What you "major in" is irrelevant. The question is what you learn. You can get a degree without learning anything, and you can learn a heck of a lot without being in college.
BTW, I think the "funny application that actually works quite well" in this WTF is supposed to be Enterprise Manager. It's kind of slow to load the guy's nice database, but it works fine once it's loaded!