Forums
After more than two years at WTF Inc., I thought I'd seen everything that could be done wrong actually done wrong in the worst possible way. Whether it was DBAs who wcouldn't administer a database if their lives depended upon it, managers who wcouldn't manage anything, or business people who simply could not understand the concept of save a dollar today, spend ten tomorrow to fix it.
After that dalliance, I'm back in my chosen field. While crazy things sometimes get done in insane ways, it's usually in the name of beating the competition to market, and (almost) always with the understanding that it will be fixed later - at a price.
However, this one struck me as sooo wtf that I'm not even going to try to anonymize it.
Every project from hello world on up has a source tree. It might be as simple as a single directory with one or more source files, or it could be an entire hierarchy of packages, common and external libraries, and so forth. The one thing they all have in common is that the one or more main programs in a project are all applications that pertain to that project. You never see two unrelated projects sharing the same source tree. They might share a common shared library, but not the same source tree. It just isn't done.
Or so I thought.
At my present firm, the culture specifies that for major architectural/code reviews, there must be one very senior member from an unrelated team/department present, and that individual has veto authority on anything that's said or presented. This is to allow an unbiased opinion to be offered without threat of reprisal by the manager.
As one of the more senior folks, I was volunteered for this task at a department that could only be linked back to me from five levels up. I didn't know any of these people, and so had no axe to grind. I went in with an open mind.
When they described their project, for the most part, their approach, level of scalability and parallelism, use of database, messaging and services, etc. made sense. Then they showed me their repository tree. It did not make sense. There were thirteen different (unrelated) projects in there. Together:
MasterRepository | |-- classes |-- srcproject1 | |-- com | | | |-- company | |-- business | |-- comm | |-- gui | |-- services | |-- util |-- srcproject2 | |-- com | | | |-- company | |-- business | |-- comm | |-- gui | |-- services | |-- util | ... |-- srcproject13 | |-- com | | | |-- company | |-- business | |-- comm | |-- gui | |-- services | |-- util
Naturally I queried why all the other projects were in there together, and why all of their source directories were configured as source directories in this project, I was told that they were told that it's easier this way.
Though I began to shudder, I just had to know, so I looked into the source trees. There were numerous classes with the same names and implementing the same interface at the same package path but in multiple source trees. Thus, auto-complete could pick any one of them because they all had the same signature, albeit subtly different implementations. As you might imagine, this led to all sorts of debugging fun at run time.If they were lucky, something would be null and it would dump the stack. If they were less than angelic, it wouldn't perform the calculation in quite the right way. If they had been particularly bad, if wouldn't perform the calculation in quite the right way only some of the time.
As if this wasn't far enough off the beaten path, I noticed that they were all using the same build/package/deploy mechanism, but at seemingly random intervals. It turns out that to prevent each of the teams from blocking any of the other teams from doing a deployment-at-will, all of the units of work were designed to be less than one day. That is, you had one day to design, code and test your work before committing. Thus, if the other team needed to deploy, they could grab the entire tree - including all of the tiny units of work done by the other teams - that compiled but didn't necessarily accomplish anything useful - package it up and deploy it.
Of course, if someone happened to be changing some piece of functionality that was shared, but hadn't yet made all of the one-day-of-work units that comprised the larger logical change, it was possible to get a melange of code that could best be described as: it might work.
Needless to say, the entire department was experiencing very high levels of instability, blocking deployment collisions when some piece of code absolutely could not be deployed without the rest of the related changes - without breaking the other projects.
When I pointed out the folly of all of this, they told me that the boss four levels up had experienced massive problems with multiple projects under his control, and decided that a single source tree would only need to be fixed once and would thus improve throughput. I told them that all of the projects had to be liberated separated into individual project source trees. Anything that was common would need to be its own project and released on its own schedule. When they were informed that this had to be done, they said they were under a mandate from four levels up. I went to the common manager five levels up, explained the situation, and that this was the reason for all the red on the dashboard.
The order to break it all apart was given.
[Advertisement]
BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!