Jan W. works in the Operations group for one of the biggest Internet access and hosting providers in Germany. As such, when you have to support as many customers as they do, there's a LOT of complex moving parts that go into keeping the environments up and running smoothly. So, to help distribute the workload, and to keep everyone sane, Operations' development needs are handled by a separate group.

Any new changes that are sent over arrive bundled up in a convenient Debian package so that QA can quickly bless everything before it's installed on any production machine.

This process is repeated, over and over, all day long, and everything flows smoothly...except for when certain exceptions arise. Sometimes, understandably, because of a critical production problem or a noisy customer, an edict is delivered from on high that an install must be fast tracked so that it skips QA and goes straight into Production.

Now, that doesn't mean that Jan and his teammates won't at least give the package a cursory inspection before pulling the trigger because, sometimes, rushed changes are sloppy and prone to system-halting errors that ruin everybody's day.

Other times, the goal of the fast track request is an attempt to sneak a hack into production that would otherwise never pass the rigors of QA inspection...like that one time when the below Perl code was included in the postinst-script of a package that runs at the end of an install.

host:# cat /var/lib/FOOBAR/FOOBAR_interval.pl
#!/usr/bin/perl
use strict;
use File::Spec::Functions qw/canonpath/; 
use File::Path qw/make_path/; use 5.006_000; 
my $conf = q{/etc/FOOBAR/FOOBAR.conf}; exit 0 unless ((-f $conf) && (-w $conf));

# Part 1: set run-interval to random value between
#         18 and 23 hours
my $base_intval = 64_800;
my $rand_offset = rand 18000;
my $interval = $base_intval + int $rand_offset; 
my $command = qq{sed -i -e 's/run-interval = .*/run-interval = $interval/' $conf}; 
`$command`;
[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!