The life of a contractor can be precarious. Contracts end- sometimes suddenly, and you rarely know what the organization you’re working for is actually like until it’s too late.
Ian S, for example, was contracting for a platform-as-a-service (PAAS) company, adding new features to their existing infrastructure automation system. It was the kind of place that had two copies of the same code-base, maintained side-by-side, just so that a single customer could use a script they’d written eight years prior.
That wasn’t too much of a challenge. The real challenge was that when things went wrong, there was almost no logging, and what little logging they got contained helpful, “[10:14:17] An error occurred” messages.
It wasn’t hard to see why that happened:
try {
// Entry point to most of the program here
} catch (Exception e) {
if ( e instanceof ProcessingException ) {
throw new ProcessingException("An error occurred");
} else if (e instanceof BatchException ) {
throw new BatchException("An error occurred");
} //… more types of exceptions
}
Ian describes this as “Pokemon Exception Handling”: you wrap the entire main method of your app in a single try, so you’re left with a single catch block that’s “gotta catch ’em all”. The use of instanceof
is a nice touch, in the awfulness of it.
The developer responsible, John, was involved in a lot of important architectural decisions. For example, John decided “DevOps” and “Agile” meant that any code placed in the production branch needed to go to production, automatically. There were no checks around this, anyone with access to the main repo could merge-and-push.
“We enforce it by practice,” John explained. “We know that all of our developers, even the contractors, will follow the best practices.”
Late, on Friday afternoon, John was working on making some configuration file changes. Among other things, his changes caused the whole program to crash on startup- but not before messing up some rows in the database. That was no problem, he was working on a branch, and running against a local dev environment.
It what John claimed was a “simple mistake”, he merged that branch with master. Then he pushed to the central repo. “It could happen to anyone,” he said. At 4:59PM, on Friday, their entire PAAS configuration and management suite went down. Garbage data was thrown into the database, repeatedly, and since there was no exception handling, the only information they had was “An error occurred.”
Truly, the life of a contractor is perilous, and for management, this became a 4-alarm, hair-on-fire emergency. All hands on deck! Even the contractors!
There was just one problem. The PAAS company had decided that they weren’t going to renew the contract. They had gone further, and announced that with a day’s notice, which left a number of the contractors flapping in the wind, between gigs, Ian included. So at 5:00PM, when he officially didn’t work there anymore, he wished John the best and went home.