The development and testing process had been rigorous, but Brody felt confident in his code after receiving UAT signoff the week before. There was even time for extensive load and stress testing this time around. Brody spent most of the weekend running builds and packaging up the code and database scripts so that Lars and Co. could prep the deployment first thing Monday.
Lars strolled in an hour late Monday morning, seemingly unconcerned with the magnitude of the release before him. "Hey Lars," Brody greeted him. "It wasn't fun, but I got everything packaged for you guys to do your thing. So you should probably get started so we can hit the ground running tonight."
Lars scoffed at Brody's urgency. "Chill out, BRO-dy. We got this. Since the last time we had a big nightmare release like this, my team created an automated deployment system. Things will go perfectly smooth, assuming you and your contractors did your jobs right."
Brody brushed off Lars' snippiness as he recalled TexOil's last big release. Everyone involved pulled an all-nighter and worked 18 hours straight just to get the troublesome release stable. In the end, Lars' team had deployed the wrong code via human error. So the fact that they had automated deployments now was music to Brody's ears.
TexOil's standard release window of midnight rolled around. Brody resisted the urging of his buddies to hit the bar earlier so he could be clear-headed for the release. Everyone signed in on time and joined the big conference call. "Ready when you are, Lars. Hit it!" Brody prepared to sit back and listen to the happenings and only get involved if things went sideways.
"Automated deployment - ENGAGE!" Lars shouted. In Brody's mind, Lars had just hit a button and a bunch of console windows were appearing then disappearing as their automated system updated everything. An hour went by without much chatter on the call. Brody assumed since the release was automated now, it would be quick. But he deduced there must be some kinks to work out.
Getting impatient and tired, Brody broke the silence, "Hey Lars. How are things looking?"
"Well, the deployment worked just fine. I think there's something wrong with your code, though. All we are getting is ASP.NET errors when trying to bring up the websites. I thought you guys said this was signed off on?" Lars questioned in an accusatory tone.
"Um, it was. Everything worked fine in our UAT environment," Brody responded with concern. "What's the error you're seeing?"
"It's just some lame generic error. It says to turn custom errors off or something. If you guys are customizing stuff, I can't be expected to troubleshoot for you!" Lars shot back.
"Just send me the configuration file, Lars. I'll take a look," Brody said, expecting something to be amiss.
"Fine, but the way our automated deployments work, there's very little chance of a configuration error." Lars attached the web.config file to an email and shot it off to Brody.
Brody popped open the XML configuration file in Visual Studio to check for the myriad changes that were required. One by one, he saw them. He started to think the configuration wasn't the problem and prepared to start debugging. But that's when he reached the bottom of the file and the distinct block of text that was Lars' e-mail signature sat there, looking horribly out of place. It was plastered with several squiggly underlines that were an obvious indicator that it wasn't valid XML.
"Lars..." Brody paused before asking a question he should never have to ask. "Why exactly is your e-mail signature in this config file?"
"WHAT?!" Lars shot back. "Give me a minute and I'll get this resolved." An awkward silence followed. Finally Lars spoke, "Ok Brody, the ASP error went away. We can have the testers hit this thing hard."
Brody breathed a sigh of relief, but immediately began to wonder how that happened. "So Lars, how did that occur when you guys have the deployment automated now?"
"Well, you see Brody," Lars started, nervous. "What we have isn't FULLY automated yet. What you package up for us, I have an app that grabs that and throws it out on the server. Then I copy it by hand to the IIS directory. I also can't figure out how to make configuration changes with it yet, so I have a cohort do those, email the XML to me and I AUTOMATICALLY know to copy and paste it in to the web.config file. It's nearly a fool-proof system though!"
Fortunately, Brody's phone was on mute so Lars and the rest of the conference call couldn't hear the stream of profanity he spewed at the sheer stupidity of this "automated" deployment.