“This,” Gregor said by way of introduction, “is Jack. Jack’s our new highly paid consultant.”

Rita shook Jack’s hand. Jack was the kind of person who entered a handshake with a dominant, overhand approach, and then applied too much pressure while he smiled at you. He wanted you to know, he was a take charge kind of guy.

“Jack’s going to set up some automated tests of the IVR system,” Gregor said.

Rita worked for a large health insurance company. Their customer facing phone system was a mission critical application. It was a programmable IVR- interactive voice response- application that needed to be able to service huge numbers of fairly routine requests. Like most things tied to the medical industry, those “routine requests” could be arcane, could be complicated with large numbers of details, and could end with something getting faxed to an office, because half the doctors’ offices they worked with didn’t exactly trust email.

“More tests?” Rita asked. She wasn’t opposed to the idea, but she had written a huge number of test scripts to walk through the logic of the back-end system. But Jack’s tests weren’t back-end tests- he had a tool that could simulate actual phone calls.

“What you’ve got is great,” Jack explained in a tone of voice that implied he didn’t think it was particularly great at all, “but what I’m going to do is better. It’s like the difference between unit tests and functional tests. We’re gonna do end-to-end functional testing now.”

Rita was the expert on the IVR system, which meant she couldn’t escape Jack. He needed her help to understand some of the testing protocols they had in place. When testing a live system, it was very important to use some special data; for example certain patient names would cause the system to spit out dummy records, which kept their tests away from live data. There were many “gotchas” to this process, which Rita had documented thoroughly, and she guided Jack through.

Jack assured her that he’d gotten it figured out, and went off to work. He deployed the expensive test package to several servers in their datacenter, doing an end-run around the operations guys and getting himself setup with admin privileges on the boxes. “We’ll never get this done if I have to send in tickets for every little thing,” he explained. After a few weeks, he pushed the “run” button, provided the results of the tests, and then, like most highly paid consultants, dried up and blew away on a stiff breeze that evening.

A few days later, Rita got a ticket, escalated from Lisa down in HR. Rita was confused at first, because Lisa had a printer issue, which wasn’t exactly in Rita’s domain, but apparently it was printing out a patient record over and over again, which caught Rita’s attention.

“So,” Lisa explained, “this is one of those big Epsons, the kind that’s printer, copier, fax, and it’s got lots and lots of paper stored away in there, and it’s burning through that paper like it’s trying to destroy the rainforest.”

“And it’s printing out a patient record?”

“Benefits stuff for one of our customers, yeah. It’s been doing it for days now.”

“Have you tried unplugging the phone cable?”

Why would a printer/copier/fax machine be printing out a patient record non-stop? Interestingly, when Rita pinned Lisa down on the start of the problem, it turned out to be the same day Jack had left.

Rita had a hunch it was the fax machine part that was the culprit, and after checking the activity on the IVR, she was certain. There was a job that was coming in with no origin phone number- it used a direct VoIP connection from inside their own network- from one of Jack’s test machines. Every three minutes, it connected to the IVR

Rita called operations. “Hey, there’s a job running on JCKRKS03 that’s causing some problems. Can I get you to kill any test scripts running on there?”

“No can do,” the ops guy said. “We’re locked out of those boxes. Zero access.”

Fine, she needed Jack to fix the mess. She called Initech Consulting.

“Oh,” the rep said, “Jack isn’t with the company any more. He moved over to Initrode.”

“Well, we still have a contract for IVR testing, right? Who’s taking over Jack’s role?”

“Oh… I don’t really know…”

The next three hours were a deep dive into a convoluted maze of bureaucracy. The rep roped in some Initech management. Rita roped in her company’s management. Initech claimed that the contract called for the test job to be set up and run- and it was clearly running. They could see the test results. Therefore the contract was completed. Rita’s management complained that they weren’t feeling like they were getting the support they had paid a large sum of money for. Fingers pointed, blame was shared around, and in the end, nobody knew what they could do short of wiping the test servers and starting from scratch, because Jack hadn’t told anyone the password.

Annoyed, Rita decided to try remoting into the box. She connected to the box JCKRCKS01. If she were a highly paid consultant, what would her password be? “password”? No. “P@$$w0rd”? No. “j4ckR0cks”? Bingo.

Rita poked around the test runner, and found that Jack had made two mistakes. The first mistake was that Jack didn’t use the correct test data. Specifically, when starting a job that would trigger a fax send, the destination number should always start with “555-”, which would never be a real number, and would cause the fax to be redirected to an email address. That mistake was a bit of a nuisance, but for the second. Under the contract, Jack was supposed to set up the tests and run them once, and then Rita or any of her peers could run them in the future on demand. Instead, Jack just put the tests on an infinite loop. The call script took about three minutes to run, so every three minutes it triggered a fax of a patient’s statement of benefits to print out from Lisa’s printer.

Rita fixed it, and double checked his other scripts. The only piece of good news was that Jack had only used internal phone numbers in all of his scripts. They hadn’t spammed an outsider’s fax machine with thousands of pages of someone’s private health information.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!