When it comes time for month-end, quarter-end, and year-end, accounts will tend to display a wide range of emotions, from cranky to flustered to downright belligerent. Nothing – and I mean nothing –is more important than getting than getting accurate financials out on time. After all, without those reports, the executives would have no idea whether they should take credit for a good period or if they need to find someone to blame for poor performance.
As a financial systems developer for a mid-size company, Mike L had come to learn that there are times to be chatty and friendly with the folks in accounting and times to avoid even thinking about stepping foot in their area. Fortunately, since he dealt primarily with back-end data aggregation, he was never in a position to get in the accounts’ way and, therefore, never had to face the wrath of accountant. That is, until some serious miscalculations occurred in WBL01GLP.
“What did you do to my report!?” Katrina said while bursting into Mike’s office. She was the financial controller and was clearly agitated. “It’s, it’s… it’s… did you even look at it?”
Having no idea what Katrina was talking about, Mike did his best to defuse the situation. “I’m sorry, I’m not sure which report you’re referring to? I generally don’t work with–”
“I know you don’t do the reports,” she interrupted, “but this is a data aggregation problem!”
“Oh,” Mike replied. He was genuinely surprised that a data aggregation bug had slipped through the cracks, as it’s blatantly obvious – even to a non-accountant – when aggregations fail. “What file is it not loading?”
“The file is loading fine. But look at this,” Katrina dropped a thirty-page printout on his desk and pointed to a group of circled numbers, “it’s really bad. My team has been doing reconciliations on these numbers for a week now!”
Mike leaned over and ran his fingers across the numbers. The job code descriptions were clearly coming through, which meant the data file was actually being loaded. The numbers looked reasonable in that they weren’t zeros, in the quintillions, or in the negative quadrillions. Clearly, the data was getting aggregated properly.
“Katrina,” Mike asked, “this looks like a report issue to me. Did you run it past Herb?” Herb was the reports developer, who used all sorts of tools from SQL to Crystal Reports to help build reports off of the accounting data.
“Of course I did,” she snapped back, talking even faster, “he said it couldn’t be a reporting problem, but he hasn’t change the report in months. The data in IRIS is correct, so that means your aggregator is broken.”
After a little more back-and-forth, Mike agreed to look at the aggregator and discover the underlying problem. Just to cover his bases, Mike gave Herb a ring and asked if he knew anything about WBL01GLP.
“Oh sure,” Herb responded, “Katrina was having some major issues with that, but I haven’t changed the report in ages. It has to be on your end. “
Frustrated that his data aggregator was to blame, Mike started digging. And digging. And digging some more. All the while, Katrina kept bugging him to see how the fix was going on. Apparently, a whole team of accountants were unable to do their job until an accurate WBL01GLP report could be generated, and none of them were happy.
Mike went back to Herb. “Are you absolutely, positively sure that you didn’t change your report?”
“Like I said, Mike,” he responded, “I haven’t changed WBL01GLP in over a year.”
Mike went back the drawing board. It just didn’t make sense that his aggregator was skipping some data and changing other data. His code worked fine locally, but it always outputted the wrong info in production. Desperate and out of ideas, he asked Herb for a copy of the report so that he might be able to reproduce the problem locally.
He opened up the report definition and noticed a three-day old comment at the top.
2007-01-15: Added in 2007 basis rate adjustment calculations
“Herb,” Mike said frantically, “you told me you didn’t change the report. But it says you modified it earlier this week!”
“Um, yeah,” Herb replied as-a-matter-of-factly, “I had to add the new rate calcs. Those change every year, you know!”
Mike stared intently. The rate calcs would have most certainly explained the incorrect figures. “So, why did you tell me that you didn’t change the report?”
“Because I didn’t change a thing,” he said, “I just added a couple lines of code at the bottom.”
After Herb removed the newly added lines, the situation all-of-a-sudden resolved itself, and the accountants were able to go back to running the year-end financials.
Mike also learned a very different definition for the word “change” that day.