1 Queue

Sandra from Initrovent, previously featured here on the site, has finally left that job, finding herself at InitAg instead. InitAg is a small agricultural tech startup, a little rough around the edges for a data science company, but overall functional. With one notable exception: The customer portal.

The customer portal is a mid-sized Symfony application, built up over the course of 5 years by an externally contracted one-man shop (let's call him Bjørn) who jealously guarded the source code up until the point he decided to drop InitAg as a client in late 2021. Upon informing them he was dropping them, he took a few weeks to clean up some of the code before giving it to InitAg. The Symfony part was mostly all right, with a few small WTFs but nothing too surprising. However, part of the portal's task was the slicing of large (>1GB) image files into smaller chunks for display in a browser, and a few other time-intensive tasks. Which brings us to today's topic: queuing.

There's lots of ways to do queuing right. If you're working in the modern day, you're probably going to use RabbitMQ or another AMQP message broker. If it's 1995, probably the Linux system queue. But Bjørn was a fan of "keeping things simple". What mechanism did he choose for the queue? A collection of "task files" (with no defined structure) in a shared directory, and a bash script on an every-5-minutes cron job polling that directory for the next task, which would then trigger other bash scripts that actually performed the long-running operation. Some of these scripts were many hundreds of lines long, and without any form of documentation or comments to explain why they were shuffling files around or performing other operations.

Of course, that's not all. It turns out that a large number of the scripts were unused (tasks that had been superseded or were no longer needed), but only Bjørn knew which ones were and weren't active.

Bjørn's reasoning for all of this? "I like to keep things simple."

It also didn't help that Bjørn had terrible git hygiene and seemingly didn't know about .gitignore, so the directory with all these scripts in it contained about 7,000 of these task files.

Thankfully, this WTF has a happy ending: Sandra was eventually able to kill off the home-built queuing mechanism and the vast majority of the scripts, and replace the whole mess with a nice RabbitMQ consumer. Still, you gotta love the guys who like to "keep things simple" by avoiding proven, robust solutions in favor of homebrew nonsense.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!