"Hey, Marcin, do you have a second to talk? Come meet me at my office. No big deal, just when you have a sec." Marcin spun around in his chair, stood up, and walked to his boss's office.
"Come in, come in," his boss waved. "And close the door."
Uh oh, Marcin thought. A "close the door" from the boss is second only to Chris Hansen saying "why don't you have a seat over there" on the things-that-indicate-the-conversation-won't-go-your-way list.
Marcin had been hired by the hosting company to design and implement a new hosting platform. Their old platform had become unmaintanable — it was a Java backend that would output generated JavaScript, much of which generated more JavaScript that would be run through eval() statements. Not to mention that it only worked in IE. The cost of maintenance became too much, so Marcin was brought on to update it to a Ruby application. He was paired with the Ruby guru they already had on staff.
After two months, work was progressing OK, but Marcin's partner, the Ruby guru, was always too busy on other projects to help. They were both working full steam, but Marcin couldn't keep up with the business's expectations, which was the reason he was now sitting in his boss's office with the door closed. When asked if he was on track with the original schedule, Marcin sighed and told the truth. "No, it's going to slip a little bit. Ryan's been reassigned to other projects, and I can't meet the original deadlines by myself."
"Well, we have to do something about this," his boss began. Marcin braced himself. He'd never been fired from a job before, and didn't know how to react, or afterwards, what he'd say about it on his résumé that he'd soon have to dust off and update.
"I want to move you to the Windows Administration team. I think that's a better place for you," he said, talking to Marcin as one would talk to a young child. "You can take the open desk near Gary's, and someone will be by later today to get you started."
Marcin considered his options, and decided to just deal with the demotion. Over the next five months, he didn't have to do much. His biggest accomplishment, a script used to migrate data from an old Windows machine onto a new Linux system, had taken him about a half day. He'd spend the rest of his free time drinking free coffee and trying (and failing) to score dates with the ladies on the support staff.
Elsewhere in the company, though, gears were in motion. Immediately following Marcin's demotion, a small team was hired to take over Marcin's task. As a part of the migration, they had to get data off of three old systems to new ones.
For the first server, it was a pretty easy task, particularly because all it required was running the script that Marcin had written already. For some reason, though, rather than run his script, the business hired a temp worker to manually migrate the data. The task involved grabbing a list of usernames and passwords, dumping it to a CSV file, and FTP the content to a new server. Using Marcin's script, this would take a half hour. With the manual process, it took six full days to get through the usernames alphabetically to "N." Marcin pushed to have his script used, but management insisted on the manual process.
The team that was in charge of the other two servers wasn't doing so well either. They struggled for weeks with getting everything running, only to give up in frustration. Migrating those servers would be moved to phase two; for now they'd just physically move the old servers.
The team was getting things done at about the same rate that Marcin was, to the business's dismay. Priorities shifted, and it was decided that they'd buy a third-party PHP app and then modify it for their needs later on.
After a few months of migrating, coding, and updating router configurations (which used the root password "toor"), etc., it was time to roll the new system out. They notified their clients weeks in advance of some brief downtime while the new systems were rolled out.
Sadly, the "brief" downtime lasted far longer than anticipated. Web sites were back up in short order, but the email stayed down for nearly a week.
The support center's phones were ringing off the hook, and the staff was working around the clock to get things running again. To deal with the calls, management had a plan — they'd streamline their process by making it only possible for one call to come through at a time. In one fell swoop, it dramatically reduced support costs and customer patience.
120 long hours after the migration started, it was complete... mostly. Email came back online, and the lines were reopened to more than one caller at a time. For the following three days, angry customers kept the support queues full.
What the business hadn't anticipated, though, is that the customer service queues began filling up — scores of customers wanted to cancel their accounts. The largest clients were the ones hurt the worst by the email outage, and they in turn hurt the hosting company the most when they cancelled their accounts. The business has yet to recover.
Otherwise, things are mostly back to normal now, though the two new servers are still untouched and the rollout is still ongoing.