It was a particularly irritating Monday morning, when Travis got a frantic text from his boss. The sun was shining, the birds were nattering, and everyone was greeting him with a smile; it was like everyone in the world had their coffee, but Travis overslept and was going to have to satisfy himself with whatever sludge he could scrape out of the office coffee maker. He had just crossed the threshold when the text arrived:

WEBSITE GONE. WHERE R U?

Travis hurried to his boss’s office by way of the coffee pot, and asked, “The website is down? Since when?”

“It’s not down!” Judging from the red face, and sweating, his boss’s blood-pressure was somewhere between “deadly” and “catastrophic explosive failure”. “It’s gone. Deleted. We think it was a security breach.”

Travis bit back the “I told you so,” he desperately wanted to say. The site in question started life as an intranet site, used by insurance agents, built with all the security consciousness that went into intranet sites circa 2004- that is to say, none. A few years later, one of the executives got themselves an iPhone, and asked the next question: “Why can’t I use this on my phone?” Changes were made, security was bolted on, and the entire development team just waited for the other shoe to drop.

They kept it alive, a twisted Frankenstein’s monster, as new technologies joined the stack, old code lived cheek-to-jowl with new code. Creaky ASP code touched the same database as shiny new NodeJS. What few tests there were had been written in the past year or so, and only touched the newest features.

Eventually, just having it on the iOS web-browser wasn’t enough. The agents needed to upload GIS data from their mobile device, and the iPhone sandbox wasn’t particularly happy about that. They needed a native app for the device. So Travis’s predecessor read a few Obj-C tutorials and whipped up an iOS app that could hold and upload geographical shape files, and present their web application.

With all the moving parts, with the complete lack of baked-in security, a breach had been a long time coming. Travis and his peers had raised their concerns, but an attitude of “it hasn’t been breached, it must be safe!” prevailed. Now they were going to pay for it.

“I’ve got Neda and Ted going through the logs,” his boss said. “Brooks is doing recovery. Work with Neda and Ted to see if you can figure out how they got in.”

Ted and Neda were the nearest warm bodies when the incident had been reported, and thus got stuck grepping through logs to find the culprit. They had come up with nothing. “There’s no sign of any intrusion,” Neda said. “Nothing unusual in the logs. Which logs we have, I guess- the server holding the web site was wiped.”

Travis left them to it, and decided to start at the beginning. He tracked down the agent who first reported the outage, Regina. Regina was based on the East Coast, and thus a few hours ahead of Travis’s West Coast office.

“My customer gave me a shapefile with some demographic data attached,” Regina said. “I uploaded, or tried, I guess, but I got an error: ‘invalid file type’. The app crashed, and I haven’t been able to log back in since.”

The shapefile was an obvious culprit. Was it compromised in some way? He had her send it to him.

There are dozens of different “standard” file formats for holding geographical data. There are open standards, like GML, semi-open (but widely used) standards, like Esri’s shapefile, and proprietary formats like MapInfo’s TAB file.

Travis’s application only supported Esri shapefiles. This was a TABfile. That, at least, explained the error the user saw. But how did that lead to deleting the entire site?

Travis started tracing through the upload process. Shapefiles (and TABfiles) actually contain many subfiles So, multiple files were actually uploaded, and they needed to be stored in a folder following the naming convention “…/username/filename”.

With that in mind, the upload process did this:

  1. Validate the upload
  2. The target directory is the same as the filename part of the file
  3. Delete the target directory
  4. Recreate the directory, and place the files there

Most of this process happened in the shiny new NodeJS code, but it still was written according to some “conventions” that Travis had been trying to stamp out. It was a mess, containing some weird logic for solving its problems.

For example, the thousand-line upload function needed to verify the file extension was correct. So what did it do? It used lastIndexOf and substr to split the filename into two parts- filename and extension. If it was correct, it put them back together by doing filename += extension. If it was incorrect- because someone had, for example, uploaded a TABfile instead of a shapefile- the code would send back an HTTP error code using response.end()… but didn’t break out of the function.

No return. No thrown exception. If someone uploaded an invalid file, they got an error message, but the file still got uploaded. Worse, since it only put the extension back on filename if the file was valid, the filename was incorrect for the rest of the processing. So, mycustomerdata.tab simply became mycustomerdata.

At this point, the bug was pretty clear in Travis’s head. He flipped to the section of code that was responsible for deleting the destination folder.

uploadedFullPathWithUsername = user.name + "/" + filename;
folderPath = path.normalize(uploadedFileFullPathWithUsername.substr(0,
    uploadedFullFilePathWithUsername.lastIndexOf(".")) + "/");
fs.removeSync(folderPath);

Since the preceding bug had “forgotten” the filename extension, uploadedPathWithUsername would have contained something like regina015/mycustomerdata. The lastIndexOf(".") wouldn’t find the target data- it’d be -1. The JavaScript substr function happily assumes you wanted a 0-length string, and thus returns an empty string. Which means, by the time fs.removeSync is called, the folderPath is a simple “/”.

In other words, this perfect storm of bad choices had reinvented the venerable sudo rm -rf $1/$2 trap.

Travis presented his findings to his boss. They hadn’t been hacked, but this vulnerability could have been exploited by a malicious user. “We dodged a bullet,” he said, “but not to mix metaphors, who knows what other land-mines are in this code?”

“But you’re saying we haven’t been breached, right?”

“Well not technically-”

“Then it must be safe! Fix this bug and get back to your regular work.”

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!