The Daily WTF: Curious Perversions in Information Technology

2023-02-21 Reply Admin

The frist change request to this MicroServiceFromHeck was to allow the users to change their username. The database rapidly went down in flames as the follow up was to add another microservice to synchronize username change across microservice databases.

nerd4sale · 2023-02-21 Reply Admin

they had to spent almost 75% more on cloud services.

Only 75% more? Truly a success then.

2023-02-21 Reply Admin

You already lost me at "it could cost them a few thousand a month to operate." How big is that "moderately powerful server"? I can get one for $100/month or so, including backup. And 4 core, 16GB server doesn't have to cost more than $50/month. Hundreds of thousands is beyond my imagination, or are we talking about pre-Euro Peseta exchange rates?

AGlezB · 2023-02-21 Reply Admin

Agreed. Even if you're an expert using the cristal ball that is Azure price calculator you'll find estimates usually fall way below the actual prices.

2023-02-21 Reply Admin

I had the pleasure of being a consultant working for an architect who wasn't anywhere near as clever as he thought he was as well.

Similar overall story: while I had the detailed technical knowhow, he made the decisions. And I was insufficiently authoritationalised to be able to overrule him when his decisions were unwise.

Entire job got canned.

2023-02-21 Reply Admin

I'll bet you ten whole fake internet points that Alvin got a kickback from the cloud provider.

2023-02-21 Reply Admin

I can sympathize with why you'd think that, but IMHO the odds are against it. Merely justifying his own existence and padding his resume with this accomplishment is sufficient reason.

2023-02-21 Reply Admin

All the joins would just be done in the application layer

Been there. The company wanted to use all the latest fads, including microservices on top of a no-SQL database. Problem is, their data was very heavily relational, so all the complicated queries that a SQL DB is optimized for had to be done in the application code instead. But by the time they scaled up enough to notice the performance problems, it was way too late to re-architect everything. I didn't stick around to see how it turned out.

2023-02-21 Reply Admin

Sometimes the reason for doing some sort of new tech is not that it makes sense for the project. Sometimes it makes sense for the person proposing the new tech, because that person is going to be looking for a new job, and having "new tech" on the resume looks good.

2023-02-21 Reply Admin

I have lived this, watched it burn, and moved on more than once. Nothing provides a better sense of schadenfreude than a small company discussing ‘big data’ and ‘machine learning’, then telling you to ‘go research it’. Consultants and execs with no clue chasing the latest buzzwords.

2023-02-21 Reply Admin

I worked IT for a medical billing company that had a CIO who treated IT a lot like this, like they were R&D. Every new thing he saw in CIO magazine he would tell upper level IT to "look into" how they could use it in-house. Also he wore very strange shoes.

He got let go after about 1-2 years of this nonsense.

Mr. TA · 2023-02-21 Reply Admin

I cannot believe that the one day my story is up, I end up opening the site in the evening :))

One addition: there is also a table with a few billion rows that has NVARCHAR columns, and a table with a few billion rows with like 30 columns, including some NVARCHAR columns (however those are nullable, which is some consolation, I guess?)

2023-02-21 Reply Admin

You are the problem. Your architect applied very standard techniques to try and dig your company out of a technical hole that you probably made.

Next time, get past the ego and push in the same direction as your employer. You might learn something.

Mr. TA · 2023-02-21 Reply Admin

Well it wouldn't be ONE server, you still need a database server (actually 2 for replication/reporting/etc.), a few web servers, most likely - a couple of big boxes with Esxi VMs, plus dedicated internet, plus backup internet, electricity... it does add up to a few thousand with on premise.

But yes, they do currently spend multiple hundreds of thousands per month on Azure. I saw the "spend report" page with my own eyes. Part of it is the silliness of "using all the toys", but another part of it is, CLOUD IS EXPENSIVE. Very expensive. Check this out:

https://www.theregister.com/2023/01/16/basecamp_37signals_cloud_bill/

2023-02-21 Reply Admin

Funny how every small business is being sold the idea that they need global redundancy in case {cloud service} drops the ball. Add in an inexperienced IT staff cranking out backups, and it’s a bit…. Predatory.

2023-02-22 Reply Admin

This is kind of my life at the moment... I'm well aware of the hazards of using microservices, but I'm also dealing with a monolith that's growing beyond any reasonable ability to maintain, and badly needs surgery to make things a bit more manageable.

WatersOfOblivion · 2023-02-22 Reply Admin

Migrating from Monolith -> Microservices is almost always an absolute mess. Mainly because doing microservices right is hard and people are learning the new shiny during the migration. Really, you need lots of integration tools like client generators and ways to organize your connectivity information (DNS names, etc.) that F/OSS tools are really lacking in.

One way that I've seen that actually does work is a "Microlith" pattern: refactor internally into "services" that are really just libraries (nested projects with Maven/Gradle/etc.) and that expose a single interface (a literal Java interface) so all of the services live in a single app server. As you're refactoring the app layer, start splitting out the database layer into multiple databases, since that's the actual hard work.

It keeps the app layer flexible and easier to implement, reduces the need for client tooling, reduces operational burden of multiple independent deployments, etc., while still gaining the scalability advantages of splitting the single database up into multiple databases to address the main scalability bottleneck and allowing you to focus on the real work. Plus, if one portion does truly need to be an independent service, you can implement version of the interface that make HTTP calls instead of in-memory calls and separating it becomes relatively simple.

2023-02-22