In the late 90s, Jeremy fought a battle against a menace more terrifying than the dreaded Y2K bug. He maintained a network management application running on Solaris which managed TDM and ATM switches, called PortLog. This prototype CMDB maintained a database of all of the equipment in the network. It created a unique identifier encoded each device’s shelf, slot and port number according to a “magic” formula. That formula needed to change in the next release, thus forcing the unique ID of each device to change as well, in every deployed instance of their database.

Ross, Jeremy’s boss and PortLog guru, provided Jeremy an update script to guide this conversion. One client volunteered to be a -guinea pig- pilot site. With appropriate permission in writing, Jeremy kicked the script off, expecting it to take 20 minutes tops. Surely just converting a bunch of numbers couldn’t take much longer than that, right? But Ross’ script kept running. And running. And…

…

…

…

running.

12 whole hours and a few cans of energy drink later, the script finally reported a successful completion. The pilot site was small, and took 12 hours to convert. How long would their much larger clients take? 12 hours of downtime wasn’t acceptable, and it was only going to get worse.

Jeremy poked through the database and found all the ID’s had been updated, but several of them had the same number, thus defeating the purpose of an ID field in the first place. Out of morbid curiosity, he cracked open Ross’ script to see what the hell it was doing. A MASSIVE stored procedure stared back at him like Godzilla peering through the window of a high-rise. Inside the procedure, there were foreach loops - one per table, once per piece of equipment. Inside all foreach loops, the key step looked like: 

ON EXCEPTION 
  UPDATE <table> SET adp_no=<new value> WHERE …; --this fixes the duplicate key exception. Runs much faster now! - Ross
END EXCEPTION WITH RESUME;

It did indeed “fix” the duplicate key exception, by forcing the value to be set whenever any exception was thrown, including a duplicate key violation. Ross didn’t care that the equipment IDs ceased to be unique. And this made it “much faster”?

Jeremy paid Ross a visit. As diplomatically as possible, Jeremy explained to him everything that was wrong with the script, and how a much more reliable, efficient one could be created before their go-live date. The new script would be much shorter, easier to support, run in minutes, and most important: actually work. Ross, of course, wasn’t having any of it.

“No way, Jeremy!” Ross bellowed back at him. “It’s too late in the game to be throwing unknown stuff at this. My script has been tested and proven to work. You said yourself that it said ‘successful’ after it was done running. That means it works!”

“Ross, this thing took 12 whole hours to run for a medium-sized client, and the data wasn’t even right…” Jeremy pleaded. “If you’d just let me…”

“It printed ‘successful’! It wouldn’t have done that if it didn’t work!” Ross interrupted. He took a few moments to catch his breath, and then found the diplomatic solution: "I’m not perfect, but since you are, you can take my existing, TESTED script and modify it. Making something from scratch is out of the question in this time-frame so get started!

Jeremy slinked back to his desk and pondered his existence. Then a light bulb burst to life over his head. He could still make the quick, simple efficient script he wanted to and just Modify Ross’ script to call it and exit immediately. All of Ross’s code would still be there, so if Ross did a cursory examination of the script, nothing would look wrong. Since Ross rarely actually did the production releases, he’d never catch on.

The next weekend, Jeremy and his fellow engineers were prepared to assault their largest client’s data with the mean script he came up with. “Are we going to be here all weekend? I heard Ross’ conversion script took 12 hours for the pilot site.” a colleague asked.

“Don’t worry, I got this!” Jeremy said confidently. “I… um… tuned Ross’s script. But, ah… let’s keep this our little secret though because if Ross finds out how lean this monster can really be, his insecurity will boil over into rage.”

They were done with their work and out at the pub before sundown. Jeremy left an email for Ross to find Monday telling him how the minor tweaks he made to the script paid off, but of course Ross deserved most of the credit. Scriptzilla had been tamed, but the code remained in the script, like Godzilla lurking beneath the sea. Jeremy feared that it would one day rise again…

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!