• Prime Mover (unregistered)

    Like, perhaps find all those places where it does it the silly way and fix them frist?

  • King (unregistered)

    This is actually the frist comment. Date handling is hard...

  • 🤷 (unregistered)

    Sorry, I was commenting on a different branch. My original comment will show up a few hours later, on a different article.

  • mama bear (unregistered)

    I worked with the code which had per-customer customizations. It's a really, really, REALLY bad idea, which leads to TRWTFs like in this article.

  • Robin (unregistered)

    Yep, I've seen that, in a previous job. Not different branches, but lots of blocks of code surrounded by conditions like "if ($customer_id == 1233)". Yes, the IDs were hard-coded "magic numbers". There were some customers (the ones who paid us the most, of course) who we would bend over backwards for and add every little feature they wanted - as a result, after being there a few months I could instantly translate the hard-coded IDs to the customer concerned. As of course could all the other developers, who had been there a lot longer than me.

  • Osric (unregistered) in reply to mama bear

    I work on this kind of stuff daily (we make gambling games, deployed on many different websites).

    It's a pain when the people handling new customers mindlessly agree to new customizations of the type that should not be made (my first reaction is often to tell them to go back to the customer, and withdraw the offer of the customization, before I get explicitly told that "just this one time", we need to comply. We always comply), but as long as the customization is sensible, I don't mind. We now have a solution with a single build of the product, where lots of GUI features can be customized through a config file, and these customizations come from our core libraries, and are shared across many different actual games. It took a while to reach a sensible design solution, but now that we have one, we can actually usually do most new customizations we get requested just through our config files, with minimal new code changes (partly because I spent a lot of time up front second-guessing everything we might need in the future for new countries, legal domains, etc).

    The solution described in this article though chills me to the bones.

  • MiserableOldGit (unregistered) in reply to mama bear

    By coincidence, I'm leafing through the source code for an app I inherited back in the mid-nineties and looked after for nearly 20 years. Apart from the shuddering horror of reviewing stuff I wrote all that time ago, it was one of these too.

    At its heart it was a good idea, but it had been developed and adapted throughout that time according to whichever customer was paying the most at the time, with some (occasionally successful) attempts to retain compatibility for the requirements of all the other customers. In fact many of the adaptations were just to make it look like it would do something in order to attract a new customer who we thought would want to see that "thing".

    At least there was no source or version control or branching or anything, it was just one, huge, wobbly mess.

  • OldTiredCoder (unregistered)

    TRWTF is seeing a time-related bug and ever thinking it would be easy.

  • Brian (unregistered)

    Been there, too. I worked at a company with similar issues - one huge codebase loaded down with all kinds of special cases for different customers baked in, so it was trivially easy to break one customer's functionality while working on something for another one. Eventually everyone got so fed up that they changed the architecture (and after a while, even the organization of the dev team) to a core+extensions model, splitting off all the customer-specific features into individual plugins.

  • Happy Coder (unregistered) in reply to mama bear

    I'm working on a codebase with customizations for 10+ customers. It can be messy at times, but if you are careful and have a good architecture, it can work pretty well.

    ... OTOH if everyone was careful, had a good setup and well educated coworkers, this site wouldn't exist...

  • (nodebb)

    Hey, look at the bright side! Each customer's version of the source code COULD be in its own zip file. But it's actually in git.

  • David Mårtensson (unregistered) in reply to OldTiredCoder

    That's one lesson you have to learn the hard way :P

  • Jaloopa (registered) (unregistered) in reply to OllieJones

    That's a tossup as to which is worse

  • (nodebb)

    Back in 2012, I booked a 4 week gig (unrelated to this story, but it got me in the door). While there the company had a disaster. Seems they had a similar setup, and one branch had a bug that caused real damage to a customer. That was bad. But "they fixed it"..only to have it pop up on one oof the clones they didn't fix. Customer wanted to sue them out of existence (to cover cost of damages).

    The situation was bad enough that I got the contract (ran 39 months) to design a single configurable (no specialized branches or clones) implementation. Good Times

  • Somebody Somewhere (unregistered) in reply to Osric

    '...after being there a few months I could instantly translate the hard-coded IDs to the customer concerned. As of course could all the other developers, who had been there a lot longer than me.'

    "I don't even see the code anymore. All I see is Globocorp, National Intergroup, Sneed's Feed & Seed..."

  • Is That Really A Crosswalk? (unregistered)

    I worked for a company which took the alternate route - if a new customer wouldn't buy without a specific feature, then it would be added to the application even if it broke functionality or workflow that existing customers used.

  • BeenHereSeenThat (unregistered)

    Ah, the good ole 1990s.

    Wait?! Its 2020! OMG.

    This is a huge pile of technical debt.

    Just yesterday, I walked by a still connected switch that at this point is up to 25 years old.

  • (nodebb) in reply to Somebody Somewhere

    I have a few IDs memorized just because we use them a lot in test scripts (e.g. the X with the most Y, for speed testing). Some magic numbers did make it into production code, but we've been switching them over to config tables (or dedicated functions, or human-readable alternate IDs, or 'declare WidgetID_GenericComment = 5', depending on circumstances. Sometimes the ideal solution gives way to 'how much other stuff do we need to build/test/deploy this month'.)

  • I dunno LOL ¯\(°_o)/¯ (unregistered)

    Spah-git-tti source code control?

    This is definitely an argument for having a warn level of "disgusted".

  • Anon (unregistered)

    Looks like the place I worked, per customer customization is a must, storing dates as UTC is a good idea because we served several customers at different timezones, except the owner didn't liked it and sometimes we had to store the local time.

    And the daylight savings thing, 15:00 every friday in daylight saving time can't be easily represented without timezone information.

  • 🤷 (unregistered)

    Yeah, I was working at a place where we would have hard coded customiuzations for our customers as well. And just like it is, at some point you just know which customer has which ID. The real fun began when we were switching from a 3 digit customer number to a 5 digit customer number. :3

    It went surprisingly smooth, but I was there only for the final rollout of the project. Apparently it had been in the works for over a year. But still, all things considered it was a very good roll-over.

    Anyway, sometimes you just don't have a choice but to hard code the customer numbers into the project, because using config files would be an even greater mess. And small companies really don't have any kind of leverage to say "no" if a customer wants their report to look a certain way. It's either do or die.

  • (nodebb) in reply to 🤷

    And small companies really don't have any kind of leverage to say "no" if a customer wants their report to look a certain way. It's either do or die.

    So, you create another report, sharing as many possible components of the original report, and subscribe the customer to the new report.

    Or... add customization parameters to the report that are named after what they do, not who wants them, and apply those parameters to that customer in the customer configuration tool/table/file/whatever.

  • Laie Techie (unregistered)

    I worked for a company which I think did an ok compromise when it comes to customizations. We had our core products as their own products (my-suite.rpm). When customers needed customizations, we would create a new product (my-suite-abc.rpm) which would provide configuration files, custom web services, custom menus, and even tie into hooks in the core application. There were obvious WTFs in the codebase (like 5 different XmlHandler classes), but the customization model was not one of them..

  • shcode (unregistered) in reply to Robin

    @Robin oh, you had conditions in code? that's nice.

    we had no source control, and the various versions were diverging by edits done on production server of each client.

    at least you still saw what the differences are. i just had to always assume nothing is the same in any of the versions.

  • Unfortunate Soul (unregistered) in reply to Robin

    Currently working on an application like this, to make things even better we made sure not to document why any of those customers had different behavior! So you just end up in a if bock with 10 possible behaviors for different customers and absolutely no clue why it's different! Fun stuff.

  • 🤷 (unregistered) in reply to Jaime

    customer configuration tool/table/file/whatever

    The configuration file is the source code in those cases. ;)

    Yes of course, in an ideal world, every code would be clean, every program would do exactly what it was designed to do and nothing more. But in an ideal world, you wouldn't need programmers because everything that anyone could possibly want from a computer has already been programmed. But the world is not ideal. Sometimes you have rows in database tables that hold more than one type of information, without a "key" column being present. Sometimes they contain nothing but a bunch of 1's and 0's and nowhere is documented that if at index 5 the value is 1 the customer gets a newsletter, but if it's 0 it means they get premium access to the customer portal. And sometimes it just starts out innocently enough with 1 report for a bunch of customers, then along comes another customer who asks "can you add this and that row to the report?" a row that no other customer has and no customer is ever going to use, except for that one customer. So you just add a small "if" into the source and be done with it. Fast forward 10 years and 3 developers later and you get to maintain code that's riddled with "if"'s for various customers, and now your boss asks to add another case for yet another customer. What do you do? Rewrite the whole application from scratch? Do you spent 1 minute, adding another if to the code? Or do you tell your boss you need 4 weeks, because you are going to rewrite the whole thing?

  • 🤷 (unregistered) in reply to 🤷

    I just noticed, that sentence near the end "Rewrite the whole application from scratch?" doesn't belong there. I originally wrote something different but decided to change the end of the post a little bit. That's what you get for not using configuration files!

  • Chico (unregistered)

    TRWTF is Scott (being new to the team) being able to decide what tickets he gets to work on.

  • (nodebb) in reply to Robin

    I like that solution better than git branches.

  • pepoluan (unregistered)

    'move fast and break things' ... I really Really REALLY hate this.

    Even Facebook had left that behind.

    I often stood my ground when other teams want to "move fast and break things".

    They all learnt that an Immovable Object (backed by CTO and CEO) is indeed immovable. And things started to stabilize and user complaints get lower and lower.

  • Some Ed (unregistered) in reply to OllieJones

    Back when I started at one of the organizations where I worked for at least a year, I encountered a production code directory full of files. There were about a dozen base filenames, and dozens of arbitrary extensions on most of them. Many of those extensions were or included things like 'old', 'bak', or 'bk'. 'brk' was a less common variant component of an extension, and the initials of the development team were also kind of common.

    I was lucky. That was the 'wish it was revision control' that I expected it to be. But the real coding horror would be, "Um, yeah, so those are the different forks of our code base for our various customers. Well, mostly. There's a few actual backup files mixed in there. We do our best to keep those to a minimum, but mistakes happen, and we can't figure out for certain when they do."

  • gnasher729 (unregistered)

    For this particular situation, the solution is lengthy but quite possible. Add a column that identifies the kind of date: Unknown, utc, or local. Then change all code anywhere that stores dates to store “unknown”. Then change all code that makes a utc/local assumption to specify that assumption, and the reading code returns the contents of the database if the assumption is met or the data type is “unknown”, and converts the date if the assumption is wrong.

    Then bit by bit change everything that stores dates to record what date they store. If two branches don’t agree, that’s fine.

    And then for heavens sake hire someone whose job it is entirely to merge these branches, and pay them very well. Your company is killing itself otherwise.

Leave a comment on “The Secret is… … … … Timing”

Log In or post as a guest

Replying to comment #:

« Return to Article