- Feature Articles
- CodeSOD
- Error'd
-
Forums
-
Other Articles
- Random Article
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
People not understanding what is data and what is presentation is the source of a number of WTFs. This isn't the first time I've seen escaped strings - escaped for one particular form of output, of course - stored in a database.
The bigger WTF here is that client side code should be able to deal just fine with unescaped strings anyway. If you use the element.textContent DOM property then the browser will handle that for you. Are they building their front end UI with string concatenation and .innerHTML or equivalent?
PS frist?
Admin
TRWTF: Frontend developers having access to modify backend code
Admin
On a project I worked on years ago, extra help was needed on the backend, so we got our frontend dev to dip into backend dev. One change he made caused some unit tests to fail, so he just deleted them. This got caught, of course, in code review, and we had a good laugh at his expense. After that, Mr. frontend went straight back to frontend dev because backend was "boring".
(I love you, frontend devs!)
Admin
99.99999% of ITs problems are of its own making and failure to learn from other disciplines.
Change management teaches us about the rules of 2s. 2 people representing upstream of the process we wish to change 2 people representing downstream of the process we wish to change 2 people representing the process to be changed 2 people on the outside the process to ask the bloody stupid questions that turn out to be REALLY REALLY important questions.
Horizontal separation of business units causes huge headaches. The attitude and culture nurtured is analogous to those people who leave small steaming plastic bags hanging from trees on a favourite dog walking route.
Admin
TRWTF is not knowing the difference between a single quote and an apostrophe ....
But seriously, when someone does something like that, and isn't new to coding, how do they look at the line of code and think "that has to be the way this problem is normally dealt with"?
Admin
Because in these days of Scrum, there's far more emphasis on getting something done quickly than in getting it done right. Forethought? What's that? As long as my tickets are closed and my velocity looks good, who cares that I've just added a ton of technical debt? Just stick it on the backlog and maybe someday get around to cleaning it up, after my little hack has poisoned the whole codebase.
Admin
I've got to agree with MIserableOldGit. How does it even occur to someone to say, "this is not a UI-layer problem"?
It is no different than saying, "we want the name of thing to display in green, so save the html with the green class name to the database."
Provide the raw data to each layer, presuming it has enough intelligence to manipulate that data in whatever form necessary (i.e., build into a form for display, build into SQL for persistence, whatever).
Admin
Sounds like that appalling bodger I used to have to work with who everybody thought was a genius, but I just ended up having to fix the broken code in the back end that he'd vomited out of his worthless brain. Worst was when I kept getting NPEs in the JSF he'd decided would be a really good idea to implement. Of course it was my fault I couldn't find the root cause. I think I quit soon after because I was so fed up with fixing other people's mess.
Admin
A very long time ago, I came across a bug tracker, written by a couple of junior programmers. They figured out they had to escape apostrophes when inserting text into the database. (Probably prone to SQL injections, but I digress) But they also escaped quotes, so every time the text was saved, every quote was doubled (" -> ""). After a few saves, you ended up with rows of quotes. It didn't take long before the application was retired. (I don't remember what replaced it.)
Admin
I wouldn't disagree, but the worst of this WTF is that they're allowed near code that deals with persistence. Cleaning up this brain-fart in the back end engine is presumably quite trivial -- just revert the change -- but cleaning it up in the database is obviously a big deal. (And also suffers from indeterminacy. Who's to know that a column entry that looks like escaped gibberish isn't actually supposed to look like escaped gibberish?)
I don't know how to solve this, though. So, you're right. Unless otherwise trustworthy, restrict FEDs to their own little silo. (It's not like I ever want to touch their stuff.)
Admin
You know, I'm pretty sure there are built-in javascript functions to handle escaping of strings.
I guess this developer couldn't decode the documentation, either.
Admin
I've done HTML-encode-on-save myself (though using standard library function to do said encoding). The idea was that (1) most user inputs for that particular site should be numeric or otherwise not-HTML, and (2) all saves go through a single class that wraps the generic steps for "connect to database and tell it to do a thing", while there are lots of display pages and every single one of them could potentially drop the ball.
A handful of stored procedures were built to expect this and decode a given input, but in retrospect it probably would have been better to let an individual save pass a flag for "don't encode this one".
Admin
Are they different keys on your keyboard? Or are you making a distinction between a single quote and an apostrophe that you're not making between straight double quotes and curly double quotes?
Admin
If you have "developers" then you are not old (at least in terms of this profession)... Real old-timers remember, programmer, coder, data entry (of the program) as distinct jobs, not to mention the plethora of analysts and architects and ....
Admin
@Brian - "Because in these days of Scrum, there's far more emphasis on getting something done quickly than in getting it done right." That is a common belief, but not actually correct. By definition Scrum is for Complex products, and the definition (in this context) of Complex is "more is unknown than is known". Put another way, it is inherently impossible to know what the "right thing" is, so the focus is on (properly) on being able to iterate.
Admin
@TheCPUWizard - Actually, You are not correct. Scrum is a method to manage people, and it is not particularly suited for design, especially of complex things.
The situation You are referring to is something which a million years ago was called "prototyping". Whenever tasked with designing something that is unknown, You created a prototype. You hacked things fast, cheaply, gluing things with scripting languages, favoring flexibility over correctness and sensibility.
You changed the prototype as the client specs changed, until everything about the problem at hand was known, and a solution could then be properly designed. Once a proper design was made, You threw out all the badly hacked code -- without pity, as it was usually garbage. The difference between the past and now, is that instead of the design-once-understood-and-recode step, the badly hacked out code is shipped.
Admin
Are you implying that what you did is acceptable because you used a (presumably middle-tier) class instead of a database trigger?!
And in retrospect you would pass a flag to prevent encoding? So, in retrospect, you would allow clients of this particular class continue to make terrible decisions on how to store data?
Admin
That ignores the reality of how many smaller shops operate where many on the team are expected to be competent full stack devs. TRWTF would be grossly inadequate E2E testing that allowed such a change into production.
Admin
How come no one has actually hit on the "correct" answer. If you want formatting and you have multiple front-ends, you need to agree on a markup. Raw HTML is a really bad idea because HTML allows way too much functionality and is a security problem waiting to happen. Anyone who says it's easy is naive and should be excluded from the discussion. Plain text has no expressiveness. So, pick one of the eleventy-billion variants of markdown and call it a day.
Also related, if this organization had any sort of code review, someone should have mentioned that the page is being built with user-supplied HTML. This is almost certainly an XSS vulnerability.
Admin
Somehow, TRWTF in this case is some of these comments.
I don't think these front-end devs touched the back-end code. They implemented their 'fix' in the form submit handler. Between where the information is extracted from the POST data and where it is handed to the back-end 'save' functions.
And for storing, the syntax is already decided based on the storage engine. SQL databases will expect string values to be presented as plain string values. Do not store any sort of markdown in a database unless you are storing a blob that you will never search by.
Admin
The frontend devs didn't have access to the backend code, they just had read/write access to the data in the database. Which is a necessary thing if you want to add new customers from the frontend or allow users to edit their account settings (pretty standard). Backend could have added validation of inputs. But in general if the frontend is really determined to do dumb things you can't stop it completely without making it useless, you just have to expect a minimum level of competency.
Admin
How exactly does a web front end access the database without going through a back end?
I hope you don't mean ...
var connection = new ActiveXObject("ADODB.Connection") ; var connectionstring="Data Source=<server>;Initial Catalog=<catalog>;User ID=<user>;Password=<password>;Provider=SQLOLEDB"; connection.Open(connectionstring);
Admin
The original WTF is text-based serialisation formats. Whenever you use in-band markers you end up with escaping madness sooner or later.
Admin
I'm just quietly disappointed that howmuchkeeffe.com (with or without the 'fe') is not a fan website.
Admin
I know what Scrum is supposed to be in theory, and I've also seen what it frequently is in practice. The two are usually quite different. It is possible to have a well-managed project with Scrum, but that takes a kind of discipline that is quite rare among many companies and coders. My point was that a lot of companies have glommed onto the concept because it's the Hot New Thing without really understanding the nuances, and some even start tying Scrum metrics into performance reviews. So now people have a personal financial stake in writing rushed, hacky code.
Fun story - just yesterday (after making my previous comment) I started working on a new task, and while digging into it I found that long ago there was some page that had an account number field on it. Someone had to add a new page that was related to that page, and decided to use the same data structure but repurpose the account number property into an object with multiple child properties, which then got serialized to JSON on the backend. But hey, it's cool, because it's just a string right? No matter that the account number in our backend and DB is now sometimes an account number and sometimes a JSON blob, depending on the value of another field. Whether that abomination is related to scrum deadlines or just laziness I can't really say, but I do know that handling it better back then would have saved me a lot of trouble today.
Admin
Not particularly, I was just noting that that particular part (how that layer did its encoding) wasn't a WTF. The larger questions (what type of encoding/decoding to do at all, and what layers to do them in), all I'm saying is that I had a couple of coherent reasons for going that route, not necessarily that it was the ideal route for the context.
Again, the vast majority of inputs to this particular thing aren't supposed to be HTML anyway. The audience for this isn't the general public, it's a few thousand users in a couple specific industries. The handful of places where we intentionally store HTML in the database have been populated by either one-time scripts run directly on the database back end (for stuff that only gets added/changed once in a blue moon), or occasionally assembled from bits of plaintext. We've just recently built a few intranet pages with rich text editors connected to intranet-only APIs (IIRC APIs have to be whitelisted to be accessible outside the intranet), and the relevant APIs do indeed pass a flag for "HTML is actually intended here, leave it alone this time" to their generic-talk-to-the-database layer.
Admin
"...and went back to tweaking CSS rules and having fights over how whether CSS classnames should reflect purpose or behavior."
How wonderfully reductive of you! Those silly UX devs, arguing over how best to create something that user might actually want to use.
Admin
No.
Agile is about letting go of the idea that you will ever envision the thing before it's built. The idea of "prototyping" comes from the physical world because we have supply chains and assembly lines to set up. Software is built from free materials with no shortages and no stocking problems.
Envisioning and then building is less efficient than building in an iterative fashion. Sometimes a full spec is necessary for contract purposes or to get funding, but it's not necessary to be successful. Agile does not favor flexibility over correctness - all code is written to production quality from day one. Some flavors of agile, such as TDD, enforce high quality as a fundamental tenet.
Anyone using agile as an excuse to continue cowboy coding is doing everyone a disservice and is "doing it wrong".
Admin
Right and wrong. Like any methodology it has an ideal in terms of procedure and process that promises to deliver, and maybe it can, but the debate on whether it can gets stifled by all the enthusiastic marketeers and inexperienced devs who took the pill.
Envisioning, as you put it, is about defining acceptance criteria, unless you do that you are presuming the client will (or can) accept whatever the dev team vomit at them. Prototyping is about trying to narrow in on those client requirements, but anyone with more than a few months doing this know the dangers.
If you've got a good team, they obtained and understood and translated requirements and delivered a good solution. But then if you had a good team, they'd probably have managed that whatever trendy nonsense was being waved around. If you have an average team and you use any one of these methodologies properly, from SSADM to Agile, and there's a half decent project manager kicking about (rare as rocking horse shit) you stand a chance of producing something acceptable. If your team is crap, you'll just deliver excuses and apologies.
I never, ever saw an Agile team producing any solution to "production quality", which isn't to say what they did wasn't "good enough", so maybe using that marketing definition of quality rather than the common-sense one. Or maybe I've been unlucky, as I'd say most times I've seen Agile tossed around, it was an excuse for cowboy hacking.
Admin
“I am old. I’m so old that, when I entered the industry, we didn’t have specializations like “frontend” and “backend” developers.” Are you kidding? I’m so old that when I entered the industry, there was no front end, no back end, no internet, no displays, and no databases. You entered your data on punched cards, and obtained you results on a printer. If you needed to preserve data from run- to-run, you wrote your data to magnetic tape, and re-read the tape next time you ran the program.
Admin
Considering their fix, yes most likely, and probaly eval to just for the sake of it.
I mean any frontend developer of quality, both knows this should be handler frontend, and would know better than to hack the backend without enough knowledge to do so correctly.
Unfortunately I have seen similar things. I have even once in the beginning stored html entities in the DB, the difference was that it was intentional since the DB was setup with Latin1, not Utf8 and we needed to store Russian and Chinese characters, and changing the database was at the time not an option.
Tog several years of careful conversion, step by step, field by field, before we had removed every trace of that atrocity :/
Admin
Fixing this mess is easy. You just go back through the audit log and -
What do you mean you don't have audit logging enabled? Yes, I know it's disabled by default. But surely everybody competent enables that as basically the first thing. I mean, how do you know whether or not your database is consistent unless you can check its consistency? How do you know who did what when unless you have a record of it?
I'm also reminded of how SQL databases tend to come with the access log disabled by default. It's like everybody is confident that poorly written queries will never happen, nobody will ever DOS attack the server, and we'll never need to come up with a reproduction of what our production workload is really like to replay it on the new system in development. We'll just automatically know all the ways our database is being used, despite the fact that it's accessible on the network by more than just a single application that we control.
Admin
It could have been even worse, but, fortunately, they forgot that they need to encode ampersands.