You have to love the folly of big companies. Bear Stearns. Lehman. Tyco. Enron. MF Global. MegaCorp. WTF Inc. They always put out advertisements telling us how soft and gentle their products are for us, our children and the planet. They cajole us with ads extolling the virtues of their products, and how we can't live without them. Of course, you know that they use the strictest rules and procedures to guaranty the safety of our personal data, and take every conceivable measure to make sure that things are done correctly. In short, we can trust them. There are rules for how to do everything. Protocols to be implemented. Procedures will be followed. Period.

Phone companies, especially, take extreme precautions when releasing software because the communications grid simply cannot be allowed to go down. Ever. I mean, it's critical that you be able to get important messages through, like: I'm on my way, or Pick up milk.

As such, it's important that every manager in every department involved in running any part of the phone system, including the call centers, can dot all the I's and cross all the T's.

Big Bob is not a real engineer these days. Mostly, he gets paid to help engineers, managers and customers communicate. His company partnered with another company that had a very rigid release schedule. Together, they built a very complex product to perform several services.

Then they had a stroke of misfortune luck; a large telco had taken an interest in the joint product and decided to purchase it. Naturally, some custom modifications were required, but BB and partners tried to remain as true to their corporate vision as possible.

The telco also demanded that they limit the frequency of their releases because the telco needed more time for their internal testing. BB and partners tried to accommodate the request.

One day, the telco started complaining that the partners' updates were causing a massive surge in support calls. Naturally, BB and partners shared their concern. However, the increase in volume wasn't unilateral; it was contained to one call center; the other call centers were unaffected. After BB made several adjustments, the telco still complained about the call volumes, but with varying levels of volume spikes, and only at the lone call center.

Eventually, an all-hands-on-deck conference call was set up so the telco project teams, procurement teams, test teams, call center managers, the cafeteria lady and BB & partner teams could hash it out.

Not long into the call, an all-out argument broke out between the call center management team and the other telco teams. Fingers were pointed. Blame was assigned. Nominees for being the sacrificial lamb were proffered:

  Call Center:             We are getting flooded with support calls at one call center!
  Capacity Planning:       We ensure that there's enough capability to move the data;
                           we don't manage what data gets sent
  Network Engineering:     Our responsibility is to move the data across the network 
                           between the sender and recipient: the messages are getting 
                           through to you; we're good!
  Customer GUI Team:       We are only generating messages that are needed, based upon
                           customer interactions; we have no choice...
  Server Development Team: Listen, if anybody from the network team would have been in 
                           attendance at the last change control meeting, then MAYBE we 
                           wouldn't be here now!
  Network Engineering:     It's not like we could go if we wanted to last week! Don't
                           forget that we were fighting a fire caused by by your team! Need
                           I rehash the domain controller issue?
  Yet Another Manager:     Cool it guys - this had nothing to do with a single network
                           load-balancer firmware upgrade. There are many irregular weeks
                           of unexplained call volume

BB had to nearly bite through his lip to keep from pouring fuel on the fire. Clearly, they had forgotten that external folks were on the call.

Apparently, the call volume had started to impact on their performance figures, and it was CYA mode all around.

After some digging, it turned out that the increase in calls was exactly correlating with an advertising campaign that had recently begun, and precisely matched the success rate of the campaign. It had nothing to do with BB or his partner company.

At the end of the call, BB and partners thanked everyone for their time and hung up. BB and the software release manager wondered about what they had just heard. The telco call center was using them as a stick to block software releases, just because the sales department hadn't accurately forecast their own success.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!