The day after I finished "Final Fantasy VII," I sat down for lunch with some friends who had also completed the game. We all enjoyed it, but felt it wasn't all it could have been. We thought we -- a group of jobless, inexperienced students with no resources -- could do better. I was the only one who had ever flipped through a book on C++, so I became lead developer. Tony, who'd never used a computer for anything except playing "The Oregon Trail," had a knack for drawing and became lead artist. Everyone else (Dan and Tim) was split up between writing (Dan) and marketing (Tim). How hard could it be?
Well, it turns out that all those people thanked in the end credits actually knew how to do stuff.
A humbling experience like that is necessary for most early developers. Stephen C. was a lot like me. There was one difference between us, though; he had a job.
Prodigy at Work
Stephen was working as a programmer for a small credit agency in the Southwest. He was building a midsize mod_perl application (a Perl module for the Apache HTTP server) to parse HTML-based credit reports for automated input into the company's reporting system. Stephen had gotten a job offer in his senior year of high school and started work immediately after graduation. Of course, his title of "Senior Programmer," which he liked, wasn't based on experience. The company was a startup, and the founder was a bit of a luddite, who thought Stephen was a "genius."
Stephen's bravado wasn't much of a burden at first. He designed and built the first version of their software in a few months, and except for a few bugs everything ran fine. Revenue was coming in and they started hiring more developers for phase two.
One component of the project was developing a module to parse HTML; a common problem with many available solutions including HTML::Parser. But that's written in C, thought Stephen. It would be too slow. Stephen didn't test to verify this, but he'd heard somewhere that calls from Perl to C were unusable due to performance problems. The founder had so much faith in Stephen, he was willing to follow any advice. Stephen would develop a custom module.
Clever Script
HyperParser was built in a few days. Stephen called it "HyperParser" because it was so fast. It was a messy amalgam of regular expressions surrounded by "clever" Perl code. Whatever, Stephen reasoned. It was written to be fast, not pretty.
The proof would be in the performance. Stephen drafted a quick HTML document and ran HTML::Parser on it. 6 seconds. An eternity. HyperParser? 0.2 seconds. Success! It was 30 times the efficiency! Stephen excitedly hacked to integrate HyperParser into the main codebase.
And the application crashed. After coding, benchmarking and writing "HyperParser" all over his notepad with lightning bolts all around it, he'd never actually looked at the output. He found the bug quickly, fixed it and noticed another bug. His main loop had no code in it. He'd tested the speed of HTML::Parser against the speed of doing absolutely nothing. By that standard, his code was efficient, but not terribly practical.
The elation of annihilating HTML::Parser's speed was short-lived when Stephen benchmarked his corrected code. Another 6 seconds. And HyperParser? He gave up after 5 minutes. Calling the thing "HyperParser" was like putting flame decals on a Geo Metro.
Angry at himself, Stephen integrated HTML::Parser into the app in a few hours. Parsing was handled by a third-party library, which meant less code on their end, and less work to maintain.
As he thought about how much worse the app would've been with HyperParser, a valuable lesson was cemented in his head: Clever coding isn't worth it. The founder learned from the experience, too. He'd put too much faith in a junior developer, wasting time and money on a module that would've been more of burden than help. Calling an inexperienced programmer "Senior Programmer" doesn't make it so.