As someone who has spent more than three decades working for all manner of huge financial-conglomerate IT departments, I've seen pretty much every kind of WTF imaginable. At every level. At every scale. For years, I chose to view it as getting paid for being entertained. But over time, it dawned on me that perhaps the reason these companies are so inept at IT is that they're so focused on the job of getting business done that they can't take the time needed to learn to think through a software development project in the way you need to in order to, well, develop software.

Chaparral Supercell 2

This time around, I joined a fairly small financial firm that has a reputation for being fairly laid back. Most of the reviews by current and former employees stated that the management allowed them the time to (reasonably) properly plan out and run a software development project. I spoke with several managers, all of whom assured me that the project was reasonably budgeted for the appropriate folks (developers, QA testers, business analysts, project managers, architects, etc.). Requirements were being mandated by an industry edict. My role was simply to be one of more than 100 Java developers on the project.

After two months on the job, it became clear that it was all a smoke screen.

Apparently, while the industry edict was fairly specific as to what needed to be done, the business analysts and architects could not agree on how to do it. You see, the architects were drawing data flow diagrams showing what data flowed between what systems, and where things would need to change in order to implement the new requirements. However, the business analysts decreed that all of the old systems needed to go away and all new systems would need to be built.

Naturally, the technical folks laughed raucously and pointed out that this would entail about 10,000 man years of effort, whereas simply augmenting all of the existing systems would take about 200 man years of development effort.

The BAs put their collective foot down and refused to allow any architecture documents to be released to any of the teams. No new data structures would be specified. No interfaces between systems would be defined. Nobody would be allowed to start work building any of the components until the BAs were satisfied that what was going to be done would make them happy.

This started about 3 months before I was hired, and went back and forth for about 5 months.

Not a single page of documentation of what the new requirements were intended to accomplish, or how it would be done was distributed to any of the technical folks.

After two months on the job, it was best summed up by this conversation (that took place over three weeks) with my boss:

  Me:   Although we don't know the specifics, we know where the industry is
        going and have a cursory idea of what needs to be done. We've built
        pipes to move generic data structures with key-fields between logical
        black boxes to perform specific chunks of processing. The details can
        be filled in later once they figure out what they want. This way, when
        they finally realize that time is short and they have to make a decision,
        we won't need to worry about the plumbing, and can concentrate on the
        business problem itself!
  Boss: This is a huge, two year project across nine departments, over 100  
        programmers and a few dozen business analysts. You are not ever going 
        to get requirements because the business doesn't know what they want, 
        but you will absolutely be held accountable for building what they need.
        If what you build doesn't solve their (undefined) business problem, it 
        will be your fault and there WILL be consequences!
  Me:   I don't mind a challenge, but I don't read minds. If they want to fly
        without a net, then they need to understand that there are going to be
        countless changes/enhancements
  Boss: They're never going to give you that leeway. If it doesn't work as
        expected on initial delivery, it's going to be a political nightmare
  Me:   I'm just a developer; politics is your department; code is mine
  Boss: You don't understand, people get fired for delivering software with bugs
  Me:   Not according to the employee reviews on glassdoor.com - something is
        fishy in this department
  Boss: HR put those there to stem the tide of folks who were refusing offers
        because of how things work here
  Me:   Wha...?
  Boss: You need to figure out what they need, and build something sufficiently
        flexible so that it does what they want, properly, on the initial delivery
  Me:   WTF?!

A very short time after that, they realized that they were going to miss the industry deadline because, as much as they didn't believe it, not all developers know all languages or how to use every tool. They had assigned front end web developers to do code to do batch processing because Java == JavaScript (they both say "Java" so it must be the same thing). They had assigned C# developers to do C++ because they're the same thing (variants of "C").

The biggest WTF was when they had financial Quant's implement a library to perform extremely complex calculations that were the crux of the changes. They wrote it in C++. Then they wrote a wrapper in WXL so that the C++ functions would be callable from Excel. Then they called the macros from the Excel spreadsheet. Then they distributed the Excel spreadsheet to the teams that needed to use the calculations. The underlying C++ was distributed only as a DLL built without debug info. When developers said that they needed to know the underlying C++ interfaces, they were told to just call the spreadsheet functions. From their server-side code.

Let's see how we might do that. I suppose they could programmatically open the spreadsheet with the embedded macros, use some library to populate a few cells representing input data, execute the macros and read the result-cells, and then close the spreadsheet without saving changes.

OBTW: this has to happen in under 1 ms because there will be many calls to this - in parallel (so I'm thinking one copy of the spreadsheet file per worker thread) - and anything slower than that will just back things up.

The more experienced folks in the department all left, leaving the entire project staffed with junior developers who don't yet see the sh*tstorm headed their way.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!