Comment On The Flat-File Society

I would love to see the code to parse that last column. I would bet its a WTF in and of iteself.

barf

Why god why.

Oh, yah, we just need to recreate a RegEx to parse out the data 'columns' in the last column there, easy enough, unfortunately its basically doing the RDBMS's job, so yah, we're gonna charge you for development of a RDBMS.

 

Pwned.[D]

I see this kind of stuff all of the time it seems like....People that
join an array into a string to put in the database, and then split it
back out to use it.  Really funny to watch them try to sort by one
of the datum in the joined field.  Heheheh.... *evil cackle*

I've been lurking here for quite a while but this was enough to respond to.  All I can say is OMFG.  What a mess.

Ken

It is this sort of thing that makes me think that sending your
developers to training is a good idea.  Especially when converting
to a new technology (Relational databases are only, what, 30 years
old?).



People get so entrenched in their way of doing things they insist that the new way match the old way -- and f'em if it wont.

Ross Day:

I see this kind of stuff all of the time it seems like....People that
join an array into a string to put in the database, and then split it
back out to use it.  Really funny to watch them try to sort by one
of the datum in the joined field.  Heheheh.... *evil cackle*

Quoth thyself with an example....

> Note that the names are Sweedish.



Actually, the names look pretty German to me. Hmmm, so maybe our economic slump isn't that surprising at all .-)

Damn - now that's some nice WTFery.

So the developers figure that that they should 'get with the times' and start using a relational db - problem is, they don't have a f'ing clue as to how to use it.

They should have really 'got with the times' and by-passed relational databases altogether and went with XML to store their data. [8-|]

*faints*

If that's Swedish then I'm an Ikea-bed. That's german, g'damn!

> Actually, the names look pretty German to me. Hmmm, so maybe our

> economic slump isn't that surprising at all .-)



Yeah, they are all German words i recognise, but none as Swedish.  For instance

the last column DATEN would be DATA in both English and Swedish.



In that column is some text such as VERSTEHEN SICH which is definitely

German and definitely not Swedish.



As far as the wtf, at least this one can be repaired with a quick script

to split the data into separate columns.  Hopefully not much is using

this table.



Then again, there are probably a few dozen other tables using this

brilliant design.

I've helped a LOT of businesses convert 3270 data streams to
client-server apps on local networks .  I understand dealing with
legacy code stuck in 80-col stone age (even IF they use DB2 or
something big on the other end), but at least most businesses at least
make the attempt to mask their dinosaur legacy app from a modern
database...  or the other way around.



I'm not sure if fooling the app is worth it, even if you can provide an
increase in data retrieval speed...  at the cost of additional
processing to make a relational database look "flat" to an old
application.



This looks like this company didn't even make an attempt to do that.  Good grief.

Yes, it is German. Some quote, order confirmation or bill addressed to
a company in the Swiss city of Zofingen (even the street address, phone
and fax number are still in). I would recommend to gray-out a few more
information, like that of the company. I also think there is a german
tax-number in it, which might identify the sender of the letter (look
for the xxxxxxxxDE6).

skicow:

They should have really 'got with the times'
and by-passed relational databases altogether and went with XML to
store their data. [8-|]

Or, better yet, use XML but store it in the relational database.
Isn't that what Microsoft is going with their newest SQL server anyway?
XML in databases? :P

Anonymous:

> Note that the names are Sweedish.

Actually, the names look pretty German to me. Hmmm, so maybe our economic slump isn't that surprising at all .-)

The names are indeed German but the CH in the addresses gives it away: welcome to Switzerland! Whatever happened to those high precision cuckoo clocks?

...They should have tried javascript *bwahahahahaha*

I actually did this in a webshop-app, and it works quite well :-)



The performance sucks huge, but it works B-)

This is no WTF this is code reusability. They already had a function to
split a string at the  tabs and they wanted to reuse it. Isn't
reusability a good thing?

I hope that was sarcasm...



In case it wasn't, that's the exact function they should have used to load data into the database in a proper relational format!

LOL.  Sure it is, but then so is having a strong-typed schema...  [:D]

our dbas preach "low column count" all the time so our bone head
developers end up having 15-20 columns + a clob w/ some xml stored in
it.  makes for some yummy code on the web server dealing w/ it all.

skicow:

They should have really 'got with the times'
and by-passed relational databases altogether and went with XML to
store their data. [8-|]

No kidding.  Where's the javascript?

WTFer:

This is no WTF this is code reusability. They already had a function to
split a string at the  tabs and they wanted to reuse it. Isn't
reusability a good thing?

Of course, if they did the RDBMs correctly, they could scrap all that code for much simpler code.



Or is my sarcasm meter off?

mizhi:

WTFer:

This is no WTF this is code reusability. They already had a function to
split a string at the  tabs and they wanted to reuse it. Isn't
reusability a good thing?

Of course, if they did the RDBMs correctly, they could scrap all that code for much simpler code.



Or is my sarcasm meter off?

Yep. It was sarcasm.

Ok.. I will use sarcasm tags <sarcasm>from know on</sarcasm>...

Anonymous:

skicow:

They should have really 'got with the times' and by-passed relational databases altogether and went with XML to store their data. [8-|]

Or, better yet, use XML but store it in the relational database. Isn't that what Microsoft is going with their newest SQL server anyway? XML in databases? :P

Don't get me started on that!  There's a guy here that REALLY wanted to do that for a while...

This WTF is SOOOOO common.  At my last job, an 11 year VB developer put a comma delimited list in a nvarchar(255) column to represent a many-to-many relationship.  I told him he really need a junction table and he said: "yeah, but that takes too long..."

This looks like an EDI message in inhouse format to me (e.g. the EDIFACT NAD segments are quite recognizable). It is perfectly ok to store such data exactly the way you sent it for several reasons, e.g.:

• archiving
• legal reasons (you may even be required by law to do this)
  
• being able to resend the message
  

hagbard:

This looks like an EDI message in inhouse format
to me (e.g. the EDIFACT NAD segments are quite recognizable). It is
perfectly ok to store such data exactly the way you sent it for several
reasons, e.g.:

• archiving
• legal reasons (you may even be required by law to do this)
  
• being able to resend the message
  

I totally agree. This looks completely like a log table you would find
for interfaces. In our healthcare apps we have an HL7 log table which
consists of (for example) a sequence number, date sent, status and the
actual message text.



This example has similar characteristics - looks like there is even some kind of batch number column there.



Nope - not a WTF at all today.

hagbard:

This looks like an EDI message in inhouse format
to me (e.g. the EDIFACT NAD segments are quite recognizable). It is
perfectly ok to store such data exactly the way you sent it for several
reasons,

Yah, I'd imagine the code that accesses this column uses it as a chunk
all the time — I don't see a way to expand it to several
columns.  Maybe they could have named it better.

Not forgetting that the decision to store the data in this way may have
been completely out of the developers' hands. Using the same flat file
approach and just replacing the file with a database will have been
much quicker and easier to implement, rather than 'doing it properly'.



I have had a few instances where I was told to do it quick and dirty.
If the difference is 1/2 day or 3 days, and the problem means the
customer is losing £10,000 a day then getting it out of the door quick
has to take precedence.

Ross Day:

Ross Day:

I see this kind of stuff all of the time it seems like....People that
join an array into a string to put in the database, and then split it
back out to use it.  Really funny to watch them try to sort by one
of the datum in the joined field.  Heheheh.... *evil cackle*

Quoth thyself with an example....

Hahaha!  You could save yourself the complexity and do:

create table THE_DATABASE ( ID integer, THE_DATA TEXT );

Haha, great "rewrite" of old system.. to be rewritten again.

Well, I maybe totally off the block here, but it depends...

Yahoo! Store stilll runs on flat files - hell, Paul Graham wrote the initial code in LISP and managed using a flat file...

 

Oh, but then not everybosy is a Paul Graham.. ;)

Well, yeah, I think Google do that as well.  There's nothing wrong with flat files, but this isn't a flat file.  It's an RDMS without the benefits.

Anonymous:

skicow:

They should have really 'got with the times'
and by-passed relational databases altogether and went with XML to
store their data. [8-|]

No kidding.  Where's the javascript?

That's in the other table.

The real WTF here is that crappy picture. :o

Okay, so it's about time I created an account (I've only posted once before... if you can guess which post it was then you have too much time on your hands.)

I assumed the crappy picture was to obscure some of the private information, though unfortunately some of it is still visible, as NoName pointed out. Unless the obscuring was done after NoName's comment, I haven't looked at it carefully enough to see whether those particular details are still there.

FuckYou, is that some kind of sex object? Well, at least you don't have to 'autorelease' ;^)

Possibly the WTF is that somebody who didn't know anything about the system saw the flat files and said, 'WTF? Flat files are bad! Put that in a database!' without realising that it's, for instance, an EDI message that they need to store exactly as they sent it. Even so, it's more of a WTF than the previous one... those true/false/true2/2/yes/maybe things are getting boring.

You know, I've been coming to DWTF for a while, and I never thought I'd be an apologetic here, but this may have a purpose (doubtful, but maybe). I recently designed a system witch a few columns and a BLOB of data for a web form. Why? Because we could never know how many columns were going to be required... only some of the data intersected all forms, and those were the actual columns. It was the only feasible design using our ancient middleware.

NSCoder:

I assumed the crappy picture was to obscure some
of the private information, though unfortunately some of it is still
visible, as NoName pointed out. Unless the obscuring was done after
NoName's comment,

No, it is still the original, totally incomplete obscuring.

NSCoder:

I haven't looked at it carefully enough to see whether those particular details are still there.

Unfortunately, they are still there: Company name, city, street
address, telephone and fax number. All still there :-( Plus that tax id
number, and also interesting details about what has been ordered (at
least an air spring and a gas spring).



The big WTF is the publishing of the private information.

And believe it ro not, there is some VERY EXPENSIVE ERP software doing
just this - and it forms the core of some of their modules.  SAP
for one has 'tables' called clusters that comprises of a few key
columns followed by a flat file version of the data. Used extensively
in their FI and HR modules to name but two.



As was noted - publishing this type of data is a far bigger WTF than anything else

> If that's Swedish then I'm an Ikea-bed. That's german, g'damn!



It is written in German. But this is Swiss Quality (tm). Look at the addresses. Zofingen, CH....

Anonymous:

At my last job, an 11 year VB developer put
a comma delimited list in a nvarchar(255) column to represent a
many-to-many relationship.

Guess that is excusable if he's only eleven.

Looks reasonable to me.

Anonymous:

our dbas preach "low column count" all the time so our bone head
developers end up having 15-20 columns + a clob w/ some xml stored in
it.  makes for some yummy code on the web server dealing w/ it all.

Speaking as a long-time Oracle dev, shoot your DBAs immediately if they
prioritise anything at all over separation of data.  You can have
about a thousand cols in Oracle, and you should happily use all of them
if you have a thousand data to store.  



Justin.

Anonymous:

Anonymous:

our dbas preach "low column count" all the time so our bone head developers end up having 15-20 columns + a clob w/ some xml stored in it.  makes for some yummy code on the web server dealing w/ it all.

Speaking as a long-time Oracle dev, shoot your DBAs immediately if they prioritise anything at all over separation of data.  You can have about a thousand cols in Oracle, and you should happily use all of them if you have a thousand data to store.  

Justin.

I hope that what his DBAs mean by "low column count" is that, generally, in a good DB design, most of your tables will have less than 20 columns.  There are exceptions and you certainly wouldn't want to violate 1NF under any circumstances.  I have trouble thinking of a scenario where having 1000 columns in a single table would be a good design.

mizhi:

Of course, if they did the RDBMs correctly, they could scrap all that code for much simpler code.



Or is my sarcasm meter off?

Sarchasm: When someone doesn't quite get sarcasm.

That last column is a good example of the "Smart Column" smell.  From Ambler's book (Agile Database Techniques):
"A "smart" column is one in which different positions within the column
represent different concepts.  Examples include U.S. zip codes, in
which the first two digits indicate the state, and client numbers in
which the first four digits indicate the client's home branchy. 
Smart columns often prove to be bad design decisions in the long term,
forcing you to make schema changes at some point."



Other "smells" include multi purpose columns or tables, redundant data, large tables, and "Fear that if you change it you might break it."



--sozin

yeah, but its not as bad as you might think.  XML data columns can
be found to a XSD specification (good), and there are XPath'ish
additions to SQL that allow you to break apart a document inside a
column.  Its pretty slick actually

Anonymous:

Anonymous:

Or, better yet, use XML but store it in the
relational database. Isn't that what Microsoft is going with their
newest SQL server anyway? XML in databases? :P

Don't get me started on that!  There's a guy here that REALLY wanted to do that for a while...

I think I can even top that. The company I work for has an object
oriented DB as it's core product, one of the customers, serializes the
objects in question, stores them in their RDBMS and retrieves them to
deserialize and use them for "queries" against our DB... :(

What usually happens is that you have the task of making a log
file.  You create a database table as above, and just spool info
as above.  You imagine that is the end of it..



And then some beancounter comes along and wants to count the beans in
that 'mine'.  Ouch!  It wasn't designed for OLAP ..



And then some consultant is called in to fix the performance problems that ensue.



And he posts a screenie off to thedailywtf.



Wtf!



Thats the way it might be ;-)