• Bill T (unregistered)

    I use to support a mainframe billing system were the inputs came from each department. Each department created the inputs using what ever program generator / report write / spreadsheet they liked.

    The solutions was a Cobol program that used a 6 dimensional array to map the input file from each department into the output format.

    After I wrote that, all we had to do was edit the parameter file each week.

    10 years after I wrote that, a buddy who was still working at that bank told me. They forced each department to use the exact same output format. But they still ran it with my program to catch any bad data before it hit the main billing system. They just edit the parameter file for the new input format. They never had to change the cobol, as far as I know that program is still running.

    FYI,,,, back in the 80's I worked for Digital Equipment Corp. (DEC)

    One of my main jobs was to convert Cobol programs from what ever system they ran, to the DEC VAX systems.

    Also did system performance optimization, It was not uncommon to take a Cobol program that took 24 hours or longer, make a few changes and have it run in 8 hours or less.

    The key was understanding that the data format in the old program was based on old storage (drum, tape), and that some of the hardware instructions on the old machines were emulated in hardware on the new machines (packed decimal instructions etc.

    Bill T

  • Buffoon (unregistered) in reply to Nagesh
    Nagesh:
    Sounds like someone tried to parse a programming language by extrapolating from a single example rather than actually consulting the language specification. (Yes, even Cobol has specifications.)

    Serves them well, I say.

    Methinks you forgot to change your name back to whatever you use when not Nageshing with the rest of them.

  • (cs) in reply to Buffoon
    Buffoon:
    Nagesh:
    Sounds like someone tried to parse a programming language by extrapolating from a single example rather than actually consulting the language specification. (Yes, even Cobol has specifications.)

    Serves them well, I say.

    Methinks you forgot to change your name back to whatever you use when not Nageshing with the rest of them.

    But he is Nageshing with the rest of them, in an effort to improve the general level of Nageshing. Just between us, I don't think it's working, but time will tell.

  • Tasty (unregistered) in reply to pjt33
    pjt33:
    Why was the client changing their COBOL code that frequently? And why did the records need importing into an Oracle database weekly? They would surely have been better off either sticking to COBOL or transitioning to a non-COBOL system rather than trying to keep the two systems running in parallel with a constantly changing interface.

    ...or transition to a COBOL with Oracle solution. The Oracle Embedded SQL suite includes ProCOBOL, and ProC to get it out later. The Pro*COBOL code can read DATA DIVISION code like the record here.

  • (cs)

    So, let me get this straight... You write a program to parse Cobol data definitions. Isn't that what a Cobol compiler does? So, you are just writing a Cobol compiler (or a small, or not so small, subset of it)? Sounds like inventing a wheel here. Someone will say "been there, done that". If you ARE writing a Cobol data definition parser, you should really look at the language specifications. They have all sorts of clues as to how these things are written.

    As to WTF's, I don't know which is worse: Cobol, or something "modern" like XML. At least Cobol has been around for over 50 years, and has some history around it, warts and all. XML seems fleeting by comparison!

  • AnotherMumpsGeek (unregistered) in reply to MumpsGeek

    I agree. A simple Cache (MUMPS) routine would easily handle that.

  • Gunslinger (unregistered)

    Frakking COBOL. But the Real WTF is customers.

  • Some Guy (unregistered) in reply to Zylon
    Zylon:
    Of course, we all know how that story ends, and five decades later, COBOL programmers are still paying for that arrogance today.

    I honestly can't tell if Alex is speaking for or against COBOL here. Clearly COBOL did enough things [b]that cost too much to replace everything still{/b] that it's still clinging to existence fifty years later.

    FTFY

  • Spewin Coffee (unregistered) in reply to Steven Seagal's ponytail
    Steven Seagal's ponytail:
    Medinoc:
    Well, I guess in both cases, the programmer of the intermediary layer expected it to work for everything the input could throw at it. It proved false, but "on paper" it would have saved everybody's time... If it worked.
    Right. If it works, you're a hero. If it doesn't work - well now you have two problems.

    This is what backups are for. You save the previous code in case it needs to be reverted.

  • C-Derb (unregistered) in reply to Some Guy
    Some Guy:
    Zylon:
    Of course, we all know how that story ends, and five decades later, COBOL programmers are still paying for that arrogance today.

    I honestly can't tell if Alex is speaking for or against COBOL here. Clearly COBOL did enough things that cost too much to replace everything still that it's still clinging to existence fifty years later.

    FTFY

    FTFY

  • fwip (unregistered)

    The real WTF is writing sed in ALL-CAPS. But I guess after a while, the COBOL starts to get to you.

  • (cs)

    (yawn) yeah yeah bin there dun that. Except back in 1998 we converted a COBOL system to a FORTRAN one. Writing a parser for the general COBOL record descriptor was rather more worthwhile than creating a separate program for each COBOL file we were given. The difference between this story and that one is that after 3 weeks the program worked like a dream and didn't need to be touched for a decade, no matter how many times the source file changed.

    Oh, and we never converted the FORTRAN program to a more modern language. It was supposed to be upgraded to a C++ / Java / whatever-bollocks system but it worked so well the customer didn't see the benefit of changing it.

  • Ted (unregistered)
    "virtual tapes"
    I think I just died a little.
  • Jerry (unregistered) in reply to Jay
    Jay:
    Someone might come to work and start shooting everyone in sight when the penalty is just life in prison or being killed by the SWAT team when they arrive to stop him. But the prospect of being suspended for two weeks without pay -- who would dare?
    Congratulations! You have plugged in to the mentality of gun-grabbers everywhere. "Yeah, there's already six laws against murdering someone, but if we add just one more that'll finally scare them off!"
  • Linus (unregistered) in reply to fwip
    fwip:
    The real WTF is writing sed in ALL-CAPS. But I guess after a while, the COBOL starts to get to you.
    SED is sed reimplemented in COBOL, for those who don't have a real OS.
  • Carl (unregistered)

    You're not a real programmer until you've written something to parse data that usually has the date somewhere near column 60 of line 3 except when it wraps to line 4.

    You're not a real man until you've written your own program generator.

    And you don't get permission to use your balls until you've realized that the program-to-generate-a-program could be optimized by simply implementing some self-modifying code!

  • (cs) in reply to Zylon
    Zylon:
    I honestly can't tell if Alex is speaking for or against COBOL here. Clearly COBOL did enough things right that it's still clinging to existence fifty years later.
    Yeah, but it completely failed to deliver on its original promise to be the language that would make professional programmers obsolete by allowing business users to write their own programs.
  • Meep (unregistered)

    "long before anyone had heard of 3rd Normal Form"

    So, Boyce and Codd revised 3NF and devised BCNF, and in fact there is a fourth and fifth normal form that all databases can be normalized to. See Chris Date's Introduction to Database Systems if you're not familiar with these terms.

    The motivation is simple: Every redundancy that you can squeeze out of your schema is a whole host of stupid Heisenbugs you'll never have to deal with.

  • PRMan (unregistered) in reply to realmerlyn
    realmerlyn:
    It could have just been a single line of Perl

    /me ducks

    FTFY

  • foxyshadis (unregistered)

    This is a perfect capture of Zawinski's Law: When you have a problem and solve it with regular expressions, now you have two problems.

    (I am not spam!)

  • iiSel (unregistered) in reply to Quango

    I'm going to have to go with Chen - 1968 on that one.

  • Pat McGee (OP) (unregistered)

    For those who suggested various solutions:

    1. This was done in around 1997. Much of what you suggested didn't exist back then.

    2. There's a lot of complexity here that I hinted at but didn't point out explicitly. Given a 10,000 line record format, how likely is it that this will fit into only one table? Or even just a few? Or that there's anywhere close to a 1-1 correspondence between the input records and the rows in various tables?

    Look back at the example of the record. See the pseudo-array, "02 PAST-JOB-CODES"? That's great for COBOL records, but anathema to relational tables. That data has to go into a separate table from the main record, and that table will get anywhere between 0 and 3 rows for each row in the EMPLOYEE-REC table.

    We ended up with around 200 tables, with lots of dependencies and cardinalities that varied between 1-1, 1-M, 0-M, and a few M-M.

    Those complexities were why most simple solutions simply wouldn't cut it.

    1. Not to mention the volume. We got around 1-10 million records a week, each record being somewhere between 10 and 1000 lines of data.

    2. Some of you expressed concerns about the requirements, saying that changing the requirements would have led to better solutions. Like changing the whole system to Oracle and dumping COBOL. I completely agree. The customer had actually tried that, spent several billion $ on it, and failed. I sure wasn't gonna try that again, not on the measly couple-million $ a year they paid us.

  • Decius (unregistered) in reply to DCRoss
    DCRoss:
    That sounds familiar.

    My first job was programming binary load lifters^H^H^H^H^H automated weather stations, each of which spit out a line of text which looked almost but not quite entirely unlike the report from any other weather station. I needed to identify all of the different variations of format and field placement, locate missing fields because some stations just didn't report everything all of the time, and somehow feed it all into a database so it could be used for weather simulations.

    It was something like getting reliable data from a GPS while someone kept unplugging it and connecting a different model every five seconds.

    Fortunately, I didn't have job security and stuck somebody else with the problem after only a few months.

    What's so hard about parsing [LOCATION ID] [type of observation: (METAR/SPECI)] [TIME]Z [WIND]KT [VISIBILITY]SM [OBSTRUCTION TO VISIBILITY/PRESENT WEATHER] [TEMPERATUTE]/[DEWPOINT] [ALTIMITER] [REMARKS]

    KBOS METAR 1652Z 05012G17KT 5SM BR 13/13 A3023

    You might have trouble parsing the remarks for the information that you consider interesting, but only if you don't have a clear idea of what you consider interesting.

  • Ru (unregistered)

    You crazy kids and your "just write a perl script to do it!"

    For a little while I worked as a technical support monkey for a company which was making language examination software. All the critical data in one particular application was held together in an inadequately secured and poorly specified interbase database.

    There was a particular, and common, database corruption for which the support team had always previously asked the developers to sort out, and getting developer time for a support issue ate up the internal budget (a WTF in itself).

    As the new and least clueless member of the team, I compared the before and after databases for one of these corruption fixes, and worked out that the actual changes that needed to be done weren't that complex. A couple of dozen lines of perl would have sorted it out a treat. I suggested that I could do this and was soundly shut down.

    Neither the application developers nor internal IT wanted to risk a non-developer writing a tool that might become critical to the helpdesk monkeys, because what if that developer left and they had to maintain it in the future? No, they would continue to require that hours of customer and helpdesk time was wasted whilst we waited for a developer to fix things, and the helldesk budget dried up a little more each time.

    I'm very happy to say that I left there very shortly after that particular rejection.

  • (cs)

    I'm surprised there's nothing for parsing COBOL data on CPAN.

  • WinDef (unregistered)

    At my first client i had to export via script information from a database for printing. It was a pain cause the "standard comapny print layout" had to be followed no matter what. And after so much pain i finaly had my meeting to present the first print out. I never forget what followed next: my boss "why do you used a script?" - "cause you told me to? remeber like 2 weeks ago?" - "no - i dont want that anymore , i want to copy & paste the information into standard ms word document" ...

    ... so yes instead of using a script over 40 people now copy paste and doing it the old school day.

  • Anonymouse (unregistered)

    They should have simply used a neural network; after an initial training periog for learning how to make the necessary changes it should be able to to them on its own...

  • Mike (unregistered)

    In COBOL, MOVE CORRESPONDING says "Hi"

  • Dim (unregistered) in reply to Tasty

    +1 It's a lot easier to insert directly from cobol to database without external solution.

  • radarbob (unregistered) in reply to brazzy
    brazzy:
    Zylon:
    I honestly can't tell if Alex is speaking for or against COBOL here. Clearly COBOL did enough things right that it's still clinging to existence fifty years later.
    Yeah, but it completely failed to deliver on its original promise to be the language that would make professional programmers obsolete by allowing business users to write their own programs.

    The original promise was not to "make professional programmers obsolete", but surely the hope was to take programming to the masses so to speak.

    In the context of the times, we're talking going from machine & assembler language with all their inherent challenges to something clearly more intuitive. The difference is far beyond today's "C++ vs C#" kind of arguments.

    As one wiser than I said "COBOL is the worst language for business applications, except for all the others."

  • Old Weather Comm Guy (unregistered) in reply to Decius
    Decius:
    What's so hard about parsing [LOCATION ID] [type of observation: (METAR/SPECI)] [TIME]Z [WIND]KT [VISIBILITY]SM [OBSTRUCTION TO VISIBILITY/PRESENT WEATHER] [TEMPERATUTE]/[DEWPOINT] [ALTIMITER] [REMARKS]

    KBOS METAR 1652Z 05012G17KT 5SM BR 13/13 A3023

    You might have trouble parsing the remarks for the information that you consider interesting, but only if you don't have a clear idea of what you consider interesting.

    Trust me. There are automated obs systems that can only fantasize about emitting METARs. I had a job similar to Decius once, and the variability among different vendors was startling, even to my jaded eyes.

    As far as I could tell, the vendors tended to offer "complies with international data format standard" as an extra-price option, and the customer did a faulty cost-versus-cost analysis and decided it would be cheaper to do a format conversion at the weather processing system rather than pay for a standard format at the observation system.

    There's an entire category of WTFs regarding data format standards that could be written from that.

  • Nagesh (unregistered) in reply to Pat McGee (OP)
    Pat McGee (OP):
    Like changing the whole system to Oracle and dumping COBOL. I completely agree. The customer had actually tried that, spent several billion $ on it, and failed.
    Sounds like the customer had bigger problems than that then (as in, they would deserve to be in this site).
  • Bill T (unregistered)

    What was your most outlandish Cobol application?

    Back in the 80's while working for DEC. I wrote a Cobol system for a small insurance company to print policies on the DEC laser printers.

    The policies were formatted using DEC Runoff, so they were binary files. Of course DEC Cobol did not officially support binary file formats.

    The way it worked was,

    1. Create an empty binary file format from the Cobol program using a system service call.
    2. extract the data from the database.
    3. add page positioning data and write it to the empty file
    4. use a system service call to spawn a "convert" command to append the policy pages and the file created by Cobol.
    5. Call a system service to send the created policy file to the printer.
  • Ryan (unregistered) in reply to Bill T
    Bill T:
    Cobol application ... I wrote a Cobol system
    No you didn't. If you were around when COBOL was around you'd know (1) COBOL is upper case (2) they were called COBOL programs not "applications".

    A "program" is short for a "computer program" -- a series of instructions for a computer to follow.

    At first, most programs were "system programs" -- code written merely to try to get the system to work. Your own print driver, or hard disk badspot handler. Because the vendor didn't supply one.

    Once you got all your system programs working, you were free to apply this newfandangled technology to a business problem. That's what it was called -- an application of technology to a problem. You "apply" a thing to a thing. You don't just "apply" and end the sentence there.

    Soon this was shortened to "system programs" and "application programs". It was still an adjective, not a noun. (Apologies to the 95% of you who don't understand English sentence structure.)

    People got tired of saying "application programs" so they just used "applications" as a shortcut. I think that's called nouning an adjective. (barf) Then it got cut further to "app". But all that happened well after the true COBOL programmers learned their craft.

  • iToad (unregistered)

    My most outlandish COBOL moment? Back in the 70s, I had a COBOL app which seemed to have stopped writing tape marks at the end of data blocks on 7-track magnetic tape. To prove this, I read the tape by hand.

    The bit density on the tapes back then was so low, that you could use a special reader to actually see the magnitized areas. It was a thin plastic strip about a foot long, filled with some sort of brown ferrite material behind a clear window. When you put it over the tape and tapped it, the magnitized bits in each frame turned black.

    I unwound some tape, put the reader on it, and started a voyage of discovery looking for the magic tapemark pattern. Indeed, they were missing, thanks to a mispunched card (we still used them) in the COBOL source deck.

  • trtrwtf (unregistered) in reply to Ryan
    Ryan:
    People got tired of saying "application programs" so they just used "applications" as a shortcut. I think that's called nouning an adjective. (barf) Then it got cut further to "app". But all that happened well after the true COBOL programmers learned their craft.

    Last night I walked into a bar with some friends, and the nice lady who greeted us asked us if we were just drinking, or if we wanted some apps too.
    All four of use are developers. All four of us were very confused for a few seconds, until we realized she was talking about food, not code.

  • (cs) in reply to PedanticCurmudgeon
    PedanticCurmudgeon:
    Buffoon:
    Nagesh:
    Sounds like someone tried to parse a programming language by extrapolating from a single example rather than actually consulting the language specification. (Yes, even Cobol has specifications.)

    Serves them well, I say.

    Methinks you forgot to change your name back to whatever you use when not Nageshing with the rest of them.

    But he is Nageshing with the rest of them, in an effort to improve the general level of Nageshing. Just between us, I don't think it's working, but time will tell.
    Improving the general level of nageshing is like Serverside Javascript... You could but why bother?

  • pantsman (unregistered) in reply to Ryan
    Ryan:
    People got tired of saying "application programs" so they just used "applications" as a shortcut. I think that's called nouning an adjective. (barf) Then it got cut further to "app". But all that happened well after the true COBOL programmers learned their craft.

    Application is already a noun!

  • Bananas (unregistered) in reply to ¯\(°_o)/¯ I DUNNO LOL
    ¯\(°_o)/¯ I DUNNO LOL:
    >But TRWTF is an actual WTF on Tuesday, amirite?! (So the by Principle of Equivalent Exchange we're going to miss Wednesday or Thursday, of course.)
    You're prescient.
  • (cs) in reply to trtrwtf
    trtrwtf:
    Ryan:
    People got tired of saying "application programs" so they just used "applications" as a shortcut. I think that's called nouning an adjective. (barf) Then it got cut further to "app". But all that happened well after the true COBOL programmers learned their craft.

    Last night I walked into a bar with some friends, and the nice lady who greeted us asked us if we were just drinking, or if we wanted some apps too.
    All four of use are developers. All four of us were very confused for a few seconds, until we realized she was talking about food, not code.

    What! In what language does "apps" mean "food"?

  • Jay (unregistered) in reply to trtrwtf
    trtrwtf:
    Ryan:
    People got tired of saying "application programs" so they just used "applications" as a shortcut. I think that's called nouning an adjective. (barf) Then it got cut further to "app". But all that happened well after the true COBOL programmers learned their craft.

    Last night I walked into a bar with some friends, and the nice lady who greeted us asked us if we were just drinking, or if we wanted some apps too.
    All four of use are developers. All four of us were very confused for a few seconds, until we realized she was talking about food, not code.

    ... Then one of us mentioned that we programmed in Perl. So she immediately called the bouncer and had us all thrown out. "This is a high-class place!" she yelled at us.

    There, finished the story for you.

  • (cs) in reply to Matt Westwood
    Matt Westwood:
    trtrwtf:
    Last night I walked into a bar with some friends, and the nice lady who greeted us asked us if we were just drinking, or if we wanted some apps too. All four of use are developers. All four of us were very confused for a few seconds, until we realized she was talking about food, not code.
    What! In what language does "apps" mean "food"?
    It's probably short for appetizers.

    EDIT: To be fair, appetizers in most bars in America shouldn't really count as food.

  • Jay (unregistered) in reply to herby
    herby:
    So, let me get this straight... You write a program to parse Cobol data definitions. Isn't that what a Cobol compiler does? So, you are just writing a Cobol compiler (or a small, or not so small, subset of it)? Sounds like inventing a wheel here.

    I presume you mean "re-inventing the wheel here". But anyway: Maybe so, but so what? What do you propose that he should have done instead? Found the source code for the COBOL compiler and extracted and repurposed the code that parsed data definitions? I rather suspect that that would have been far more work than starting from scratch.

    I think that "re-inventing the wheel" cliche is seriously flawed anyway. After all, in real life, companies "re-invent" literal, physical wheels every day. It's not like there's one wheel factory that makes one kind of wheel and that's what everybody in the world uses for every application. You do not use the same wheels on a child's tricylcle that you use on a 50-ton earth-moving machine. It is not at all unreasonable to suppose that the specific characteristics of a wheel for one vehicle might not be ideal for another. Indeed, when I recently got new tires for my pick-up I had a choice of many options, from a range of qualities to whether I wanted highway tires or off-road tires. Even for the same vehicle I might choose different wheels depending on how I intended to use the vehicle.

    Furthermore, manufacturing companies regularly consider whether to "make or buy", that is, whether it is more efficient and economical to buy something from an outside source or to make it themselves, based on a variety of factors.

    The same is true of software. Just because someone else has written, say, an accounting system, doesn't mean that it is stupid for anyone else to write his own accounting system. We might have different requirements and different priorities. Or we might just think that we can do it better.

    I often hear people say that it's dumb to write software yourself when you can download what you need from the Internet for free. But software downloaded from the internet may be available at no charge, but that does not make it free. Why would we download such "free" software rather than writing it ourselves? Surely the cost being considered is the time of our employees. But does it take zero time to download free software? No. Someone must take time to search the Internet to find candidate products. We must take time to read the descriptions to see if this sounds like it meets our needs. Then we must download them and install them on our local computer. We must learn how to properly configure the software. We must learn how to use it. Then we have to study the details of its features to see if it really does meet our needs. If not, we may be able to customize it, but that means getting the source code and decyphering that. If it doesn't meet our requirements and we can't customize it, then we have to start the process all over with another product.

    In some cases, of course, that is well worth the effort. In other cases it would be easier to just write what we need ourselves. I would need very unusual requirements before I would write my own database engine. But if I need a function that I know I can write and debug in a couple of hours, I'll just do it.

  • (cs) in reply to Decius
    Decius:
    What's so hard about parsing [LOCATION ID] [type of observation: (METAR/SPECI)] [TIME]Z [WIND]KT [VISIBILITY]SM [OBSTRUCTION TO VISIBILITY/PRESENT WEATHER] [TEMPERATUTE]/[DEWPOINT] [ALTIMITER] [REMARKS]

    My question at the time was "What's so hard about reporting in that format?", since much of the data we received didn't follow it.

    But that became somebody else's problem decades ago.

  • C-Derb (unregistered) in reply to trtrwtf
    trtrwtf:
    Last night I walked into a bar with some friends, and the nice lady who greeted us asked us if we were just drinking, or if we wanted some apps too. All four of use are developers. All four of us were very confused for a few seconds, until we realized she was talking about food, not code.
    It wasn't until I read the word "food" that I realized she was talking about appetizers. I was trying to remember if I'd heard of a new kind of bar that sells alcoholic drinks and smart phone applications. When's the IPO, amirite?
    PedanticCurmudgeon:
    EDIT: To be fair, appetizers in most bars in America shouldn't really count as food.
    +1
  • C-Derb (unregistered) in reply to Jay
    Jay:
    My understanding of that "re-inventing the wheel" cliche is seriously flawed....
    FTFY
  • JJ (unregistered) in reply to Ryan
    Ryan:
    Soon this was shortened to "system programs" and "application programs". It was still an adjective, not a noun. (Apologies to the 95% of you who don't understand English sentence structure.)
    Count yourself in that 95%, then. "System" and "application" are both nouns which are part of a noun phrase. While they are describing another noun, they are not adjectives; rather they are simply considered "modifiers."
  • Bill T (unregistered) in reply to iToad
    iToad:
    The bit density on the tapes back then was so low, that you could use a special reader to actually see the magnitized areas. It was a thin plastic strip about a foot long, filled with some sort of brown ferrite material behind a clear window. When you put it over the tape and tapped it, the magnitized bits in each frame turned black.

    You had the fancy toys. We just dipped the tape in an oil with the ferrite material shaken up in. The ferrite would stick to the tape. We would then put scotch tape over it, pill off the scotch tape then look at the bit patterns.

    About two years ago I was cleaning the basement and I found my card reader. This was a metal frame that you would set the IBM punch cards are to assist in reading them by hand.

    I Pitched it along with my 132 column print layout ruler, and my mini guide to DEC COBOL Syntax.

    I did keep a copy of the DECUS (Digital Equipment Corp User Society) magazine that had a copy of a COBOL program I wrote to submit a file to the printer.

    I would say that is all that is left of my COBOL days

    Bill T

  • Sigivald (unregistered) in reply to Steven Seagal's ponytail
    Right. If it works, you're a hero. If it doesn't work - well now you have two problems.

    As the Unix Hater's Handbook said:

    Some people, when faced with a Unix problem, think "Aha! I know! I'll use sed!" ... now they have two problems.[/i]

  • Alvie (unregistered)

    WTF ? This guy never heard of Perl ?

Leave a comment on “The Program-Generator Program”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article