Anyone with any significant amount of experience has had to estimate a project of some complexity. The only real way to do it is by breaking down the project into major parts. Then breaking each part into smaller parts and so on, until you have a list of units-of-work that you can reasonably estimate the amount of time that will be required to do that work. Then you figure in dependencies, see what can be done in parallel, factor in available staffing, add it all up, pad by as much as you think you can get away with to account for unscheduled changes, miscalculations, emergencies and management stupidity. Finally, you put it into a project management tool and make your presentation to the Powers That Be.
Rob worked for a company that brought in three consultants to do a major rework of their database structure. One of them was being paid about $85 an hour for the sole job of filling out and maintaining a Microsoft Project schedule for the project. The first thing the consulting team did was put together a proper schema diagram of the database to be reworked. Then they tracked down all of the code that wrote to or read from each table. Armed with all of this ammunition, Mr. Project was able to put together a project time line.
Fast forward to the meeting where the project plan was to be presented to the bosses. Mr. Project stands up and proudly drops a stack of paper (that looks like an entire Amazonean village was defoliated to create it) onto the conference table with a resounding thud. He announces that this is the project plan. As he runs through the executive summary - just to list the major work areas (problem analysis, work estimates, resource scheduling) - the brass all nod along in agreement. It was standard stuff. Then he started to get into bottom-line numbers.
He explained that in order to come up with first-pass estimates on how much work was involved, they chose a methodology that involved analyzing a few database tables, and then multiplying by the total number of tables involved to get rough estimates.
MrP: It will take something like 35 full time developers an entire year (70,000 hours), just to analyze the database C**: <cue collective gasps> MrP: Once that part is done, we can assess the amount of work required to rewrite code... CEO: There's no way we can allocate 35 people just for this CIO: Even if we did, all other work would grind to a halt; we have commitments to our customers CTO: Even if we didn't, we don't have enough infrastructure to support that size project just for this purpose
Rob was somewhat taken aback by this. Knowing that their database was in dire need of help, but not that much help, Rob asked him how many tables were involved. The answer was over 1,000 tables. Rob laughed and asked to work with him off-line to attempt to refine that estimate a bit.
Once back at their desks, Mr. Project showed Rob all the tables in the schema. There were, in fact, more than 1,000 tables, but more than 900 of them were named like this: TMP_65239423756893, and they had been accumulating for quite some time.
Ok, Mr. Project didn't use any common sense and was just doing a quick calculation on the total number of tables in the list. But the most fascinating part of all of this is that when Rob went back to the suits to advise them that the estimate would be more like a few man months, he found them having an all-out management panic trying to figure out how they could possibly allocate that many people to just one phase of this project, and how they were going to pay for it...
CEO: If we cancel the Flugle and Anaconda projects, we could save enough to fund this CIO: Maybe, but those projects support direct commitments to our major customers; that is not an option CTO: To do this, we'd need servers for developers, database servers, drive arrays, I/O-switches, a lot of server room floorspace to put it all, additional network bandwidth, and a cubicle farm big enough to hold four normal-sized development teams CEO: Do we really need to clean up this database that badly? Can it wait? CIO: Project lead-times have significantly increased in the past two years. The development managers have repeatedly said it's because the database was never intended to handle our rate of growth; waiting will mean more expensive software development going forward ...
The suits didn't even seem to realize just how bizarre this all was...