Chinese buffet2.jpg
"Chinese buffet2". Licensed under CC BY-SA 3.0 via Wikimedia Commons.
Assembler. C. C++. C#. PHP. Javascript. Bash. Perl. Ruby. Java. These were just some of the technologies featured on the resume of a candidate Christian recently interviewed for a senior Linux sysadmin position. The impressive list of programming languages (and related data-interchange acronyms like XSLT and JSON) made the candidate, let's call him Rob, seem more qualified for a developer position, but he went on to list common web server databases like MySQL and Postgres (plus a couple flavours of NoSQL), and, finally, the qualifications Christian was actually interested in: Tomcat, JBOSS, the Hotspot JVM, and every major Linux distro. While the resume reeked of keyword-baiting, Christian didn't want to risk missing out on an excellent sysadmin who just happened to spend a lot of time hacking, and brought Rob in.

Christian kicked off the interview by describing their infrastructure. Working for a major enterprise, his division was responsible for fifteen hundred Java application servers, clustered into groups of three or four. He explained to Rob how they managed the large number of identical deployments using Puppet, with SVN to track changes to their enormous catalog of scripts. He got through most of their rollout and monitoring processes before Rob cut in with a question.

"You mentioned Subversion earlier. You developed this in-house?"

Christian stared at Rob for a while, trying to understand the question in the context of the seasoned developer's resume on the table in front of him. He realized Rob was getting a little uncomfortable, still waiting for his answer.

"We, um... No. No, Subversion is an open-source version control system. It's, well, quite popular. You... hadn't heard of it?"

"Not at all," Rob replied. "Version control, you said?"

"Yes. Sometimes also called configuration management software. Perhaps you have more experience with git, or mercurial?" Rob shook his head. Christian tried moving in the other direction. "CVS?" Another shake. "Er... Visual SourceSafe?"

"No, I'm afraid not. Version control sounds very interesting, though. I look forward to using one!"

The interview went careening downhill from there. Christian turned to some questions about Linux, and Rob's sysadmin training turned out to come mostly from Googling how-tos. Questions about processes and threads, memory management, and—most importantly—the JVM, flew over his head. Nearing his wit's end, and desperate to salvage the interview, Christian asked Rob to describe one of the web applications he claimed to have built, hoping to discover an unusual insight or something, anything, that would account for the gap between the resume and the human.

"I once took an order-management application and ported the whole thing from MySQL to PostgreSQL."

Interest piqued, Christian leaned forward. Could this be Rob's redeeming point? Maybe this was where he could offer value to his team.

"That's a pretty major change," Christian said. "What prompted you to make it, and how did you carry it out?"

"Well," Rob opened his hands, as though the answer were obvious. "MySQL didn't work anymore."

"You mean... what, it didn't scale?"

"No, I mean the service wouldn't start anymore."

"The... mysqld?"

"Yeah, somehow it broke and wouldn't start anymore. I tried init.d and rebooting, but it didn't help. So, after I uninstalled MySQL and installed Postgres instead, it worked."

Christian pressed for more details. "Okay... but changing the DBMS must have had a big impact on the application, right? MySQL and Postgres are very different, after all..."

"No," Rob said, "I mean, come on, they're both SQL. It wasn't that big of a change."

"Well... how did you plan the migration at least? This was an order-management system, right? So you'd have to do it in a way that preserved order history."

"Migration? We didn't do that," Rob said. Now it was his turn to fix Christian with an incredulous gaze. "I told you, MySQL wasn't working, so I uninstalled it. The data was gone, obviously. We entered it again from scratch."

This raised more questions than it answered—no backups? What about transferring the MySQL DB to another machine? Did he at least post the log file on StackOverflow with a terse "why doesn't this work?"—but by this point, Christian was desperate to get back to real life. He complimented Rob on his extensive experience and told him they'd let him know if he got the job. On the way back to his desk, Christian tore Rob's resume into pieces, one for each technology he'd pretended to understand.