Comment On eTeller Horror

Most large applications are designed with multiple, autonomous interacting components. In very high level terms, they look like this:

The two circles above are two mutually dependent components. The arrow is an avenue of communication between the two, and two classes can only communicate where they are connected to one another. In general, you want there to be as little 'surface area' between two classes as possible, and the avenue of communication should be as narrow as possible. Neat, simple, orderly, supportable.

Now consider the following:

This is how the eTeller application supported by Steve looked under the hood.

Prices So Low, They're INSANE!!!

Infotech is a company that produces budget enterprise software for various financial institutions. Sure, in this context, "budget" may be defined as being in the neighborhood of a generously equipped new car. But in the eyes of a bank, even a small one, spending this kind of money is a bargain when compared to writing and supporting something written in-house. So, to remain competitive and keep their prices appealing, Infotech, like many companies with similar goals, bottled up their requirements and offshored their development.

The same process was applied to Infotech's latest initiative, eTeller, which was a suite of applications used by bank tellers at their stations to keep track of day-to-day things like deposits, withdrawals, receipt printing, and the like. After a year of development, eTeller 1.0 was released and, thanks to a base of existing satisfied customers, the money rolled in. However, their growing customer base was quickly turning from a blessing into a curse.

The "Fit" hits the "Shan"

Working on the front lines, Steve's group of support analysts were well aware that the product they supported was, by definition, crapware. Six months after the initial release, eTeller was up to v1.5 and had gained a following of thoroughly disgruntled customers. Every day, more and more users were calling in about the frequent application crashes mid-transaction, inaccurate reporting that lead some banks to think their tellers were robbing them blind, and several other nasty things were not just keeping them from going about their daily business, but also making them seriously ticked off!

After apologizing in a soothing tone, analysts in the US would send a list of new bugs to the offshore developers, who would then address the problem and release a patch which, when installed by the client, would open up three new bugs in its place.

With customer satisfaction at an all-time low and bank presidents calling Infotech's CEO, at home, threatening to sue, something had to be done. The contract with the offshore developers was terminated and the source code was recalled with the plan being to fix the bugs and make enhancements locally at the only-slightly-higher-than-offshore "greenhorn" rate.

The developers complied, but nothing could have prepared Steve, the CEO, or anyone else for what they were about to inherit.

The Beast Revealed

Source analysis revealed that eTeller was over 100k lines of tangled, messy, and completely unmodularized C# code...and not a single line of it held any redeeming value whatsoever. It was so bad that refactoring it would have probably made it worse; the whole thing just needed to be rewritten.

A few of the "highlights" included:

• Excluding classes generated by Visual Studio, there were all of five classes used in the entire application. Each was a static class containing exclusively static methods. The main class, "Globals", contained 1000s of variables to maintain the state of application. The only types used in the Globals class were bool, string, decimal, and DataSet.
• As a result of the lack of classes, no business-specific classes existed in the application. Instead, ArrayLists, HashTables, and DataSets were used to move around data.
• 20,000 lines of code were embedded in a single file called frmMain.cs. It printed receipts, performed account inquiries, controlled the cash dispenser, you name it!
• No primary keys, indexes, or foreign key constraints existed on tables, nearly all fields were of type varchar(50), and 100% of fields were nullable.
• Who needs stinkin' parameters when you could store everything in the Globals class? In the rare instance that parameters were passed to a method, they were passed as strings for convenience.
• Some forms had invisible labels. The programmers used the .Text and .Tag properties to hold miscellaneous data. Invisible labels must have been easier to create than inserting yet another variable into Globals.

Of course, because they had to support customers who were reaching for their last straw, a full rewrite of eTeller had to be put on hold. Banks were demanding to have bugs fixed on the same day they were reported...or else. Each new bug the development team into crisis-mode: bugs were being fixed as fast as possible and a new release was being sent to QA almost immediately.

Against the recommendations of QA and developers, new versions of eTeller were being released to customers every two to three days. Fortunately, this was only a "temporary" operating mode and, once things cooled down, they'd scale back a bit and focus on the big, rewrite project. At least, in theory.

Many, many months later, when they finally delivered the arbitrarily-numbered eTeller v1.89b.6a, the system was at a state where it performed its most basic tasks with minimal errors. There was still a steady stream of bugs coming in, but one thing was certain: a rewrite was never, ever going to happen.

Taming The Beast

Steve's team was faced with a foe they could never defeat. Every time they worked to slay a bug, the codebase would grow and become an even larger monstrosity. Instead of going head-to-head with the beast, they decided to take a less ambitious approach: surgically remove dead features and rewrite functionality whenever a change request came through.

The approach seemed to work, though it did have it drawbacks. Not only did changes take longer to implement, but the codebase didn't seem less ugly; it was only a bit smaller. Sure, the once 20,000-line frmMain.cs had shrunk by 60%, but was still an 8,000-line mess.

Eventually, their careful, calculated approach caught up with them. When it came time for a major new set of features, there was no way Steve and his team could maintain their pace of bug fixing and deliver the changes in a timely manner without hiring more developers.

Fortunately, managers, in all their wisdom, knew exactly what to do. And all Steve’s team would have to do is bottle-up a few technical requirements...