It was Friday, but the atmosphere at Andy’s firm was far from celebratory. The launch for their new product line was a flop, with long website outages resulting in customer complaints and slumping sales. The cause was easy to pinpoint: the ETL job for the new product line’s database took upwards of 30 minutes to run each time, causing timeouts and locks from which the rest of the infrastructure couldn’t bounce back.

Development and rollout had been entrusted to a consulting firm, and their CEO received an irate phone call. “Not to worry”, he said. Their expert would fix this.

Not much later, Andy, a clutch of nameless middle-managers, business analysts, and nosebleed-level executives found themselves rounded up in the office’s largest conference room, face to face with the expert, Nigel. Nigel leaned back as far as the broken task chair allowed, and rolled a pen through his fingers in a way that seemed to defy basic laws of mechanics. Andy knew nothing about the implementation of the new product line, but he did know something about databases.

“We’ve gotten a jump on evaluating alternate ETL tools,” Andy’s boss said. “Andy here can make an early recommendation to save you time.”

Andy presented a spreadsheet with statistics he’d compiled on the three tools they’d scrambled to get eval licenses for. “Here are some test runs, using a few thousand records from other databases. All of them execute in a matter of millis-”

Gold Medal

“Excuse me.” Nigel sat up and leaned forward, wearing a look of amused incredulity. “We put the original solution in place. We’re not going to drag-and-drop someone else’s software and call it a day. That never works. You don’t want to be paying that license fee the rest of your lives, do you? Don’t worry- I’ll handle this.” He cracked his knuckles, then stood and made his way to the door.

“Where the hell are you going?” someone blurted.

“To the nearest open office. Did you want me to fix this, or sit in meetings all day?”

Nigel left a wake of murmurs behind himself. He was a dismissive prick, but Andy respected the desire to get started now rather than let the fix languish amid hours of estimate-building and platitude-making. He wished he had the clout to wander out of ineffective meetings.

Andy’s respect faded, as days, then weeks, dragged on without a new solution in place. Indeed, Nigel spent a whole month writing a new custom loading process, which he built directly into the web code, as a single 3,000 line method. His efforts reduced the load time from 30 minutes to 10.

Not ideal- but the website started behaving itself, and that was all management cared about. That, and being able to say they’d boosted performance by 66% without having to burn any more of the company’s money or time. The higher-ups churned out a bevy of congratulatory PowerPoints, and hailed Nigel’s “unconventional” genius.“It’s an impressive improvement,” they declared.

Ten minutes seemed rather long- the new product line couldn’t involve moving that much data around, could it? Andy wondered just how big these transfers were. He tracked down the job history.

None of the jobs handled more than 3,000 records at a time.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!