Comment On The Program-Generator Program

When you've been in IT for as long as Pat McGee, you're bound to have survived at least one or two COBOL horror stories. While COBOL is certainly not the worst platform to develop software on (MUMPS will most certainly hold that title through at least our grandchildren’s lifetimes), its extreme verbosity and unique idiosyncrasies make it a challenge for organizations to develop clean, maintainable code. [expand full text]
« PrevPage 1 | Page 2 | Page 3Next »

Re: The Program-Generator Program

2012-06-19 10:14 • by Chronomium (unregistered)
Kind of like the inner-platform effect, except more parallel.

So "Motorcycle-Sidecar-Effect".

Re: The Program-Generator Program

2012-06-19 10:17 • by snoofle
article:
...at least he had job security

...until the assignment would unexpectedly end.

Re: The Program-Generator Program

2012-06-19 10:42 • by Steven Seagal's ponytail (unregistered)
The real WTF is the attempt at a general solution to the problem (lexer, parser, SED script, C/C++ code - really?) when just keeping a diff between customer revisions and manually making the change would be simpler and less error-prone. Instead of simply making the change the customer wants, they have to fix their program and massage the input data to get it to come out as expected.

I understand the inclination to come up with a general solution, but the added layer of indirection can be a huge pain in the ass if you don't do it right. I have a coworker who does this with our app configurations, and instead of me just adding a line in a YAML file like I used to be able to do, I have to go in and patch his stupid program to accommodate it, then update all of our other apps that use the program to get the latest version. And he thinks this is saving us time and effort (and wonders why I get grumpy about it)...

Re: The Program-Generator Program

2012-06-19 10:46 • by PiisAWheeL
383448 in reply to 383446
Steven Seagal's ponytail:
The real WTF is the attempt at a general solution to the problem (lexer, parser, SED script, C/C++ code - really?) when just keeping a diff between customer revisions and manually making the change would be simpler and less error-prone. Instead of simply making the change the customer wants, they have to fix their program and massage the input data to get it to come out as expected.

I understand the inclination to come up with a general solution, but the added layer of indirection can be a huge pain in the ass if you don't do it right. I have a coworker who does this with our app configurations, and instead of me just adding a line in a YAML file like I used to be able to do, I have to go in and patch his stupid program to accommodate it, then update all of our other apps that use the program to get the latest version. And he thinks this is saving us time and effort (and wonders why I get grumpy about it)...


At least you have job security.

Re: The Program-Generator Program

2012-06-19 10:47 • by Quango
"...started back in the 1960s – long before anyone had heard of 3rd Normal Form... "

Defined by Codd in 1971, so no they probably would not have heard of it. :)

Re: The Program-Generator Program

2012-06-19 10:57 • by Nagesh (unregistered)
Is it really that hard to parse that? It looks like a simple field-length mapping.

Sorry, I mean, I am not being seeing a problem here in Hyderabad.

Re: The Program-Generator Program

2012-06-19 10:57 • by Axeman (unregistered)
Ah, The good old days eh?

Re: The Program-Generator Program

2012-06-19 11:00 • by MumpsGeek (unregistered)
Shouldn't be too hard to handle with a simple Cache (MUMPS) routine.

Simples!

Re: The Program-Generator Program

2012-06-19 11:12 • by Anketam
This would not have been half as bad if the customer had a standardized format for reporting the changes. However changing the format of import data that frequently is generally a bad idea.

Re: The Program-Generator Program

2012-06-19 11:13 • by realmerlyn
It could have just been a single Perl program.

/me ducks

Re: The Program-Generator Program

2012-06-19 11:20 • by pjt33
Why was the client changing their COBOL code that frequently? And why did the records need importing into an Oracle database weekly? They would surely have been better off either sticking to COBOL or transitioning to a non-COBOL system rather than trying to keep the two systems running in parallel with a constantly changing interface.

Re: The Program-Generator Program

2012-06-19 11:26 • by Scrummy
"COBOL-format record and record descriptors don't respond to change very well..."

Only because they were built using archaic development methodologies.

Re: The Program-Generator Program

2012-06-19 11:28 • by Zylon
Of course, we all know how that story ends, and five decades later, COBOL programmers are still paying for that arrogance today.

I honestly can't tell if Alex is speaking for or against COBOL here. Clearly COBOL did enough things right that it's still clinging to existence fifty years later.

Re: The Program-Generator Program

2012-06-19 11:29 • by @Deprecated
383463 in reply to 383460
pjt33:
Why was the client changing their COBOL code that frequently? And why did the records need importing into an Oracle database weekly? They would surely have been better off either sticking to COBOL or transitioning to a non-COBOL system rather than trying to keep the two systems running in parallel with a constantly changing interface.

That was my thought too.
What kind of production system database changes that fast, and in so many unexpected ways?

Re: The Program-Generator Program

2012-06-19 11:31 • by C-Derb (unregistered)
I had a job while going to college that was very similar to this. We were receiving data files from 1000+ customers, usually in Excel or Lotus format. There were 4 of us who spent all day manipulating these files into a fixed width format to be imported into our database.

One week I got really grumpy about it, vented to my coworkers, vented to my roommates, cursed our clients while driving to and from work.

Then I got my paycheck and realized it was exactly the same no matter what my attitude was. Things got better after that.

Re: The Program-Generator Program

2012-06-19 11:36 • by David Emery (unregistered)
He should have dug around the Open Source community. There's a bunch of code in Ada as part of the GNAT (open source) Ada compiler that interprets COBOL PIC statements (that I helped write about 15 years ago.) With that code, he could have fed the PIC statements and generated the associated read/write routines, reusing the design even if he didn't want to link the Ada routines in to C or C++ (another feature of Ada...) The syntax for PIC clauses is quite static, you should capture the grammar/semantics of COBOL once.

But someone show me a similar big fat ugly messy data structure definition in some other language. Do you think that struc definitions for this data in C would be less readable? He should be grateful for COBOL's ability to represent this structure, even if it was un-normalized....

Re: The Program-Generator Program

2012-06-19 11:37 • by steve (unregistered)
http://sourceforge.net/projects/cb2xml/

there are many standard copybook translators

Re: The Program-Generator Program

2012-06-19 11:37 • by Sayer (unregistered)
383467 in reply to 383464
C-Derb:
I had a job while going to college that was very similar to this. We were receiving data files from 1000+ customers, usually in Excel or Lotus format. There were 4 of us who spent all day manipulating these files into a fixed width format to be imported into our database.

One week I got really grumpy about it, vented to my coworkers, vented to my roommates, cursed our clients while driving to and from work.

Then I got my paycheck and realized it was exactly the same no matter what my attitude was. Things got better after that.


So the problem remained, but you just chose to not let it get you down.

Re: The Program-Generator Program

2012-06-19 11:44 • by AB (unregistered)
When did Pat become James?

"When you've been in IT for as long as Pat McGee,..."

ABCD123456MCGEE JAMES PACCTAR ABCD65432104250CLRK010195INTN010397

Oh I see it now, Pat is his middle name and he's one of those weirdo's that uses their middle name rather than their given name.

Re: The Program-Generator Program

2012-06-19 11:45 • by Medinoc
383470 in reply to 383446
Steven Seagal's ponytail:
The real WTF is the attempt at a general solution to the problem (lexer, parser, SED script, C/C++ code - really?) when just keeping a diff between customer revisions and manually making the change would be simpler and less error-prone. Instead of simply making the change the customer wants, they have to fix their program and massage the input data to get it to come out as expected.

I understand the inclination to come up with a general solution, but the added layer of indirection can be a huge pain in the ass if you don't do it right. I have a coworker who does this with our app configurations, and instead of me just adding a line in a YAML file like I used to be able to do, I have to go in and patch his stupid program to accommodate it, then update all of our other apps that use the program to get the latest version. And he thinks this is saving us time and effort (and wonders why I get grumpy about it)...

Well, I guess in both cases, the programmer of the intermediary layer expected it to work for everything the input could throw at it. It proved false, but "on paper" it would have saved everybody's time... If it worked.

Re: The Program-Generator Program

2012-06-19 11:47 • by markfiend
My first proper coding job was a bit like this. Not quite COBOL but I was pulling data out of text files that had not-quite-fixed-width columns and numeric-fields-except-when-they-contain-a-letter and all sorts of similar exceptions.

To complicate things further it was a two-dimensional set of rules (the client's customer's address will be in columns 60-80 on lines 5-10, that sort of thing) that had originally been designed to be printed by a dot-matrix printer onto pre-printed labels. I have no idea why whatever-it-was that produced those text files couldn't have been changed to send the data straight to the database but hey, it was a job.

I did what I could to keep it maintainable, and to be fair I've never had my former employer ring me up asking for help. AFAIK my script is still working away on those horrible text files.

Addendum (2012-06-19 11:55):
customer's address will probably be in columns 60-80 on or around lines 5-10

Re: The Program-Generator Program

2012-06-19 11:47 • by ¯\(°_o)/¯ I DUNNO LOL (unregistered)
>At the heart of the system was a
Mmm hmm...
>10,000 line
okaaay...
>COBOL
ummmm...
>record descriptor
(TURNS WHITE AS A SHEET)


But TRWTF is an actual WTF on Tuesday, amirite?!
(So the by Principle of Equivalent Exchange we're going to miss Wednesday or Thursday, of course.)

Re: The Program-Generator Program

2012-06-19 11:50 • by Steven Seagal's ponytail (unregistered)
383473 in reply to 383470
Right. If it works, you're a hero. If it doesn't work - well now you have two problems.

Re: The Program-Generator Program

2012-06-19 11:51 • by Steven Seagal's ponytail (unregistered)
383474 in reply to 383470
Medinoc:

Well, I guess in both cases, the programmer of the intermediary layer expected it to work for everything the input could throw at it. It proved false, but "on paper" it would have saved everybody's time... If it worked.

Right. If it works, you're a hero. If it doesn't work - well now you have two problems.

Re: The Program-Generator Program

2012-06-19 11:52 • by C-Octothorpe
383475 in reply to 383467
Sayer:
C-Derb:
I had a job while going to college that was very similar to this. We were receiving data files from 1000+ customers, usually in Excel or Lotus format. There were 4 of us who spent all day manipulating these files into a fixed width format to be imported into our database.

One week I got really grumpy about it, vented to my coworkers, vented to my roommates, cursed our clients while driving to and from work.

Then I got my paycheck and realized it was exactly the same no matter what my attitude was. Things got better after that.


So the problem remained, but you just chose to not let it get you down.
No, he went to work the following day with his favourite automatic weapon...

Re: The Program-Generator Program

2012-06-19 12:03 • by C-Derb (unregistered)
383476 in reply to 383475
C-Octothorpe:
No, he went to work the following day with his favourite automatic weapon...

I would never condone that, but if you must, please shoot yourself first to make sure the weapon is working, then shoot others.

Re: The Program-Generator Program

2012-06-19 12:15 • by MBV (unregistered)
So why didn't he write a COBOL program that fed the COBOL structures to Oracle? Seems like a lot easier to me, as Oracle has an interface for COBOL.

Re: The Program-Generator Program

2012-06-19 12:16 • by nB (unregistered)
383478 in reply to 383458
realmerlyn:
It could have just been a single Perl program.

/me ducks
Honestly, I think that's what I would have done. I did something similar when we had a particular program at my lab that needed a massive struct{} for register map that changed from project to project. It was always given in an xls file (8 megs or so) and used to be manually transcoded to c structs. I wrote a perl script to do it. My job role was allocated two weeks for that activity, my script did it in 40 seconds. I got a promotion out of that :/

Re: The Program-Generator Program

2012-06-19 12:17 • by Anderson Silva (unregistered)
383479 in reply to 383446
Steven Seagal's ponytail:
...
Less talky, more front-kicky.

Re: The Program-Generator Program

2012-06-19 12:19 • by ur (unregistered)
383480 in reply to 383476
C-Derb:
C-Octothorpe:
No, he went to work the following day with his favourite automatic weapon...

I would never condone that, but if you must, please shoot yourself first to make sure the weapon is working, then shoot others.

Surely it's simpler just to carry a spare gun?

Re: The Program-Generator Program

2012-06-19 12:20 • by Nagesh
DATAPUMP ==> is feature of Oracle. It will allow you to pump data from any source.

Re: The Program-Generator Program

2012-06-19 12:35 • by Dave (unregistered)
383485 in reply to 383446
Steven Seagal's ponytail:
The real WTF...general solution to the problem...manually making the change would be simpler and less error-prone...they have to fix their program...inclination to come up with a general solution....added layer of indirection...huge pain in the ass...get grumpy about it...


Loving all the "well that doesn't sound like a particularly good way to do things" comments today.

It's The Daily WTF, guys. That's kind of the point.

Re: The Program-Generator Program

2012-06-19 12:39 • by myName (unregistered)
Two real WTFs - not using COBOL to write the translation program, and changing to Oracle.

Re: The Program-Generator Program

2012-06-19 12:51 • by PedanticCurmudgeon
Did Pat actually know COBOL, or did someone just show him how to read record descriptors?

Re: The Program-Generator Program

2012-06-19 12:57 • by Mason Wheeler
So what the article is basically saying is that his sufficiently complicated C++ program ended up containing an ad hoc, informally-specified, bug-ridden, slow implementation of half of COBOL? :P

Re: The Program-Generator Program

2012-06-19 13:07 • by Andrew (unregistered)
383489 in reply to 383446
Steven Seagal's ponytail:
Instead of simply making the change the customer wants, they have to fix their program and massage the input data to get it to come out as expected.
Feke Nagesh detected!

Re: The Program-Generator Program

2012-06-19 13:09 • by MindChild (unregistered)
383490 in reply to 383452
Nagesh:
Is it really that hard to parse that? It looks like a simple field-length mapping.

Sorry, I mean, I am not being seeing a problem here in Hyderabad.


Yes, it is. You see, the currency field will have commas... sometimes. And the record is comma delimited. The middle name field will... sometimes... have a char(13) instead of a space. And so on

Re: The Program-Generator Program

2012-06-19 13:16 • by Slapout (unregistered)
383491 in reply to 383478
nB:
realmerlyn:
It could have just been a single Perl program.

/me ducks
Honestly, I think that's what I would have done. I did something similar when we had a particular program at my lab that needed a massive struct{} for register map that changed from project to project. It was always given in an xls file (8 megs or so) and used to be manually transcoded to c structs. I wrote a perl script to do it. My job role was allocated two weeks for that activity, my script did it in 40 seconds. I got a promotion out of that :/


How long did it take you to write the script?

Re: The Program-Generator Program

2012-06-19 13:17 • by Sayer (unregistered)
383492 in reply to 383476
C-Derb:
C-Octothorpe:
No, he went to work the following day with his favourite automatic weapon...

I would never condone that, but if you must, please shoot yourself first to make sure the weapon is working, then shoot others.


I used to work at a fairly prominent Telecomm and we were so inundated with procedural (and managerial) stupidity that my coworkers would joke that if anyone ever showed up one day with a gun, all the other employees would be annoyed that they hadn't been notified and had left their guns at home.

"Christ! It's gun day?! Why did no one tell me?! Do I have time to run home and get mine?"

Re: The Program-Generator Program

2012-06-19 13:41 • by d (unregistered)
Kill it! Kill it with fire!!!

Re: The Program-Generator Program

2012-06-19 13:48 • by Geoff (unregistered)
383494 in reply to 383466
That was my first thought as well. Surely there had to be some simpler way to figure out what record type you are working with and use the correct copy book ahead of time, and just version the copy books.

Re: The Program-Generator Program

2012-06-19 13:58 • by Coyne
383495 in reply to 383449
Quango:
"...started back in the 1960s – long before anyone had heard of 3rd Normal Form... "

Defined by Codd in 1971, so no they probably would not have heard of it. :)


Like that matters. These people were like Mel: Even if they'd heard of it, they wouldn't have used it. It would have infringed on their freedom. "Besides, how would you optimize your data retrieval?"

(I know: I were one.)

Re: The Program-Generator Program

2012-06-19 14:03 • by Jay (unregistered)
383496 in reply to 383475
C-Octothorpe:
No, he went to work the following day with his favourite automatic weapon...


That wouldn't have worked at my last company. They had an Official Company Policy that any employee who brought a gun to work was "subject to disciplinary action, including suspension or termination". Someone might come to work and start shooting everyone in sight when the penalty is just life in prison or being killed by the SWAT team when they arrive to stop him. But the prospect of being suspended for two weeks without pay -- who would dare? I could stand 30 years in prison as long as I knew that I could get my old job back when I got out, but wow ...

Re: The Program-Generator Program

2012-06-19 14:09 • by Jay (unregistered)
383497 in reply to 383466
steve:
http://sourceforge.net/projects/cb2xml/

there are many standard copybook translators


In fairness, if the original poster's problem was that he couldn't get his translator to successfully translate a COBOL layout to a C struct, then one could fairly say that that isn't really that difficult a problem. Sure, maybe his first pass he forgot to handle OCCURS clauses or "V" in format specifiers or whatever, but you'd think with a few iterations he could get that worked out.

But I presume he had to do some sort of processing. I mean, it's not enough to just have the C struct, you have to actually move the data across. That adds another level of complexity. Was it just a straight data move, copy "name" to "name", "account number" to "account number", etc? Or did he have to figure out record types, do data translations, etc.

Re: The Program-Generator Program

2012-06-19 14:13 • by RichP
383498 in reply to 383458
realmerlyn:
It could have just been a single line of Perl p̶r̶o̶g̶r̶a̶m̶.

/me ducks


FTFY

/me ducks too.

Re: The Program-Generator Program

2012-06-19 14:18 • by Nagesh (unregistered)
383499 in reply to 383496
Jay:
C-Octothorpe:
No, he went to work the following day with his favourite automatic weapon...


That wouldn't have worked at my last company. They had an Official Company Policy that any employee who brought a gun to work was "subject to disciplinary action, including suspension or termination". Someone might come to work and start shooting everyone in sight when the penalty is just life in prison or being killed by the SWAT team when they arrive to stop him. But the prospect of being suspended for two weeks without pay -- who would dare? I could stand 30 years in prison as long as I knew that I could get my old job back when I got out, but wow ...

Idiot. Ain't policy said about shooting: only about posession.

Re: The Program-Generator Program

2012-06-19 14:30 • by Nagesh (unregistered)
Sounds like someone tried to parse a programming language by extrapolating from a single example rather than actually consulting the language specification. (Yes, even Cobol has specifications.)

Serves them well, I say.

Re: The Program-Generator Program

2012-06-19 14:41 • by trtrwtf (unregistered)
383501 in reply to 383496
Jay:
C-Octothorpe:
No, he went to work the following day with his favourite automatic weapon...


That wouldn't have worked at my last company. They had an Official Company Policy that any employee who brought a gun to work was "subject to disciplinary action, including suspension or termination". Someone might come to work and start shooting everyone in sight when the penalty is just life in prison or being killed by the SWAT team when they arrive to stop him. But the prospect of being suspended for two weeks without pay -- who would dare? I could stand 30 years in prison as long as I knew that I could get my old job back when I got out, but wow ...


I know, right?
'cause the only reason anyone would ever bring a gun to work would be to shoot the place up...

Re: The Program-Generator Program

2012-06-19 14:46 • by Sales Geek (unregistered)
383502 in reply to 383452
Looks fine because it's a simple example. Wait until you see three pages of REDEFINE segments.

Re: The Program-Generator Program

2012-06-19 14:52 • by DCRoss
That sounds familiar.

My first job was programming binary load lifters^H^H^H^H^H automated weather stations, each of which spit out a line of text which looked almost but not quite entirely unlike the report from any other weather station. I needed to identify all of the different variations of format and field placement, locate missing fields because some stations just didn't report everything all of the time, and somehow feed it all into a database so it could be used for weather simulations.

It was something like getting reliable data from a GPS while someone kept unplugging it and connecting a different model every five seconds.

Fortunately, I didn't have job security and stuck somebody else with the problem after only a few months.
« PrevPage 1 | Page 2 | Page 3Next »

Add Comment