Credit: gralm @ flickrFor as long as David B could remember, stories of The Phantom Duo were a popular pastime around the watercooler. Of course, no one had actually worked with them, but everyone knew someone who knew someone that did. And the tall tales told why.

As the story went, The Phantom Duo were corporate boogeymen who reported to no one. They’d scour the project charter, looking for healthy, on-time, and on-budget projects that they could latch-on to. Once they were assigned to a project, it was too late: anything they’d touch would crash and burn, taking everyone else with it. Obviously, since anyone who worked with The Phantom Duo would no longer be working, only hearsay and rumor supported the legend.

For the most part, The Phantom Duo served as a light-hearted response to why projects fell behind. “Yes, we’ll hit that milestone a little late,” project managers would often say, “but at least we’re safe from the you-know-who.”

A Phantom Menace

In his four years with the company, David had little to fear from The Phantom Duo, as every project he worked on was behind-schedule and over-budget. Considering that he worked in the medical device software division, and especially considering that he was on the MRI software team, it was par for the course.

Even his latest project was following woefully behind, despite delivering the first prototype ahead of schedule. The problem, as usual, was external factors. And this time, it was a truly baffling API for a third-party library they were using to parse proprietary messages from their MRI machine.

Most third-party libraries – especially the exorbitantly expensive ones that David’s company would buy – come with extensive API documentation that explains the various methods and datatypes contained within. But this particular library shipped only with a few outdated snippets of example code. Normally, that wouldn’t be a huge problem, but the library itself exhibited a few “problems”.

Instead of using arguments when calling functions, the library required that dozens of mysteriously-named public arrays be populated in a very specific way. Errors and exceptions encountered by the library code were mostly ignored and occasionally logged to a randomly-named file in the current working directory. Worse still, an incorrect call to the library would lead towards a dialog window popping up that simply read “Error!”, which was especially problematic since there would be no user present to click the ok button.

After wrangling the library’s quirks for a few days, David decided it’d be much easier to write his own parsing library. Interfacing with medical devices was no walk in the park, but it was much better than moseying through the library’s minefield. His boss, Jared, agreed and took the strategy change to the project’s planning meeting.

“Actually,” one of the technology VPs responded, “I’m pretty sure that’s not a third-party library. It was developed for some other project a while back; perhaps the original developers could shed some light on it… or at least help find the source code?”

The VP’s suggestion was spot-on. Despite being unusable, the library’s source code could prove to be a valuable resource in decoding the cryptic hardware messages. And as luck would have it, not only was the source code available, but so were the gurus that developed it. With David’s help, they could make a parsing library to be proud of.

The Phantoms of the Library

“You introduced a lot of risk to this project by using Visual Studio 2008.” The voice seemed to come from nowhere and everywhere at the same time. Turning around, David was even more startled to find a tall, vaporous figure standing in his cubicle. He quickly rubbed his eyes to reveal a dark-skinned, bald man with a stoic glaze locked on him. Before he could even respond, the man started talking again. “2008 is not fully supported by central IT.”

“I uh, ummmm,” not having any clue who this fellow was, David found himself a loss for words, “err, it’s on... err… limited phase-in... support.”

As David struggled to force those words out, a much shorter man slid into view from behind the taller man. He shared a dark complexion but sported a full head of disheveled hair.

The tall man continued in an unemotional tone, “it was not appropriate to have this project be 2008. It is medium risk, and therefore should have been in 2005.”

As he lectured, the shorter man quickly shook his head “no” whenever 2008 was mentioned, but slowly nodded “yes” for 2005.

“Okay, but we got approval for 2008 and prefer using 2008; why is it an iss–”

“Our library is certified for Visual Studio 2005, and not 2008.” the taller man interrupted while his companion continued serve as a life-sized bobble-head on the 2008 and 2005 cues. He tightened his lips and widened his eyes, commanding “you will need to convert the project from 2008 to 2005.”

“But why?” David still wasn’t sure who these two were, and was starting to get annoyed, “that doesn’t make any sense! It’s just a library; it doesn’t even matter.”

“I’m not here to place blame,” the man offered a fake smile, “but this project doesn’t need this risk.”

David leaned back in his chair and closed his eyes for a moment to carefully plan his next words. “Look,” he said, sitting up, “I’m not sure who you–” He stopped himself. The pair was no longer there.

Phantom Operations

The mysterious men turned out to be the hardware gurus that were special-ordered for the project. Not only did they both have PhD’s in engineering, but they were the only two people in the world who knew the arcane inner-workings of the API they developed. As such, they were as critical to the project as a PMP-certified project manager.

Everyone on the team soon became accustomed to how the gurus operated. The tall one was managerial while the shorter one exclusively wrote code. Practically joined at the hip, the manager guru would do all the talking while the coder guru would simply nod or shake his head.

The two would never show up to meetings on time, instead coming inappropriately early or unfashionably late. Or they’d just materialize completely unexpected and unscheduled, whenever and wherever they pleased. Returning the favor was an impossibility, as no one really knew where the gurus’ cubicles were located. Some believed they were on the 6th floor, a handful were certain it was the 2nd floor, others said the 13th floor, and a few swore they worked out of the Poughkeepsie offices.

The Phantom Plan

“We need a plan,” Jared quietly said as took a bite of his sandwich. He nervously looked over his shoulder just to make sure that the two missing team members didn’t decide to join them on a late lunch at Chotchkie's. “They’re billing more hours to this project than all of us combined, and they’re blaming us for ‘interfering’ with their ability to write code.”

“Are you kidding me!?” David feverously gesticulated, nearly knocking over his glass. Everyone’s eyes widened, fearing the outburst might somehow summon a certain unwanted attention. Realizing the faux pas, he leaned in and whispered, “we only required that they update before commit! They refuse to merge anything, and our changes keep getting blown away.”

The group sat in silence for a few moments, reflecting on how difficult their day-to-day had become over the past few months. And then one of the developers offered a suggestion: “couldn’t we just send them to The Lab?”

It was a brilliantly simple idea. So simple, in fact, that it’s no wonder that it was overlooked before. Not only did The Lab house a fully-functioning MRI machine, but it was two states away. It was usually reserved for hardware-level diagnostics, but occasionally software developers found it useful, especially for really tricky bugs.

A Phantom Ticket

It wasn’t right, but it had to be done. Field testing of messaging timing loop fails B80C, B809, and B8AA; cannot reproduce in simulated environment. David hit “submit ticket” and, moments later, felt an unmistakable ominous presence surround him.

“That can’t be possible,” Dave turned his head to find the tall guru sporting a scowl and the short developer shaking his head, “that must be an error in the diagnostics system.”

“It does seem strange, but I ran a full diagnostic last night. I guess I’ll head to the Lab whe–“

“Actually,” the guru cut him off, “this is a library issue. We’ll go.”

“Okay,” Dave responded with a discreet smile. The plan seemed to be working. “Let’s see when they’re available.”

He quickly logged-on to the Lab’s schedule and then turned his head back to see when they’d be available. Not surprisingly, a thin, black fog outlined where the gurus were a moment ago.

And they stayed gone. No emails, no surprise visits, and best of all, no check-ins. The team was relieved and, after a few weeks, the project’s official risk level shifted from “danger red” to “concerning red-orange.” After another month, it was at “worrisome orange”.

The gurus were still billing an obscene amount of hours to the project, but at least they were spending their time chasing a phantom bug instead of bugging everyone else. The plan worked beautifully.

Phantasmal Explosion

Perhaps it was a slow news night, or perhaps it was just that bizarre of a story. MRI Explosion Evacuates Hospital Wing. David caught the clip on the 11:00 newscast.

For most people, such a story would elicit a chuckle and a hmph, I didn’t realize MRI machines could explode. But for those who work closely with MRI machines, the knee-jerk reaction is please don’t let it be our MRI and please don’t let it be our fault .

MRI explosions are serious business. In addition to utilizing an obscenely-powerful 3.0-Tesla magnet, an MRI’s superconducting coil requires a good 1,500 liters or so of liquid helium to be kept near absolute zero (-454°F). A several-hundred degree jump to room temperature will cause whole lot of evaporation pressure, which is usually ventilated by a series of rupture disks and blowout panels. But ice buildup and other conditions can cause safety measures to fail, leading to shrapnel taking out solid-brick walls.

When David returned to work the next day, he was welcomed with some bad news: the MRI machine was in fact theirs. All told, $2.9M in equipment was destroyed and an additional $5M or so in damages were anticipated. As the day progressed, the bad news kept coming. It was installed by his company. It was recently re-certified. There was no ventilation blockage. The ventilation subsystem worked fine. The quenching subsystem failed to activate. The pressure management subroutines crashed.

And then came the worst news of all: the culprit was the monitoring system, and that was recently updated on-site by David’s team. Apparently, two engineers – a rather tall, dark-skinned man and a quiet, stocky one – visited the hospital for routine maintenance and to debug a message timing loop issue.

The details beyond that were sparse, but the consequences were very clear: the company announced that it was shifting development of its MRI software to an entirely differently division in an entirely different country. That meant a “significant reduction in headcount” was needed, and David, Jared, and the rest of the team were first in line on the chopping block. As for the mysterious hardware gurus, no one had seen or heard from them again.

In the years since, the “MRI Incident” has faded into a tall tale from long ago. It’s told every now and then on a hot summer day around an ice-cold watercooler, but it’s nowhere as legendary as the stories of The Phantom Duo. One guy even swears that he once worked for a guy that knew a guy named “David” who actually worked with The Phantom Duo.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!