The sound of the phone woke Sergio from a deep sleep.

"We've been hacked. All of our data is gone. I can't believe this is happening"

Gathering up all of the wits he could find at 2:15 in the morning, Sergio contemplated his next move.

"Who are you?"

"It's Peter." A beat of silence. "Peter from Communibox"

That was enough to trigger a string of memories. Peter was the CEO of Communibox, a distributor of telecom hardware. About five years back Sergio had done some contract work for them. Nothing much. Just the website, inventory system, CMS and a risk management assessment. Peter himself was a fairly intense, alpha personality. A visionary, at least in his own mind, he was full of ideas and surrounded himself with people whose job it was to implement them while he moved on to the next big thing.

"Ah...Peter. What happened?"

The downside of some "visionary" personalities is that once a problem has been solved, there is no need to revisit it. After all, it's already working, so why waste time on it. There are newer and shinier objects to play...er...work on. So despite Sergio's requests at the time, Communibox never cared to maintain the application, upgrade the physical hardware or even patch PHP or MySQL. Eventually, Sergio stopped contacting them, comfortable in the fact that he had done a good job and that the customer was happy. That's a win-win in most consultant's books.

"Everything. It's all gone."

"No problem Peter. I'll check and call you back as soon as I know something"

Sergio cracked open his laptop, found the connection information (see...backups do come in handy sometimes) and tried to log in. Not surprisingly, his old credentials worked like a charm. As he went through the applications, Sergio found that the CEO was right. The vast majority of the data was gone. Not everything, mind you. But where there once was tens of thousands of products with description, images and real-time stocking information, there now was only a hundred or so entries left. The result was that Communibox's website, a fairly typical on-line catalog, was nearly empty.

The next obvious destination was the backups. The idea didn't leave Sergio brimming with confidence. He recalled a conversation he had with Peter while performing the risk assessment.

"Peter, you need to back all of this data up. Your catalog is growing like crazy and you need to protect it."

"I've got it covered", said Peter. "We're resellers of a very expensive backup solution and our expert is putting it in place for your system."

"All I really need is a dump of the MySQL database and some zipped file," countered Sergio. "A costly system isn't worth the effort. The key is that it needs to be off the server and preferably off prem as well."

"Not to worry. I've taken care of it"

Sergio reached out to Peter to get the name of the 'expert'. Turns out the he was the sales rep for the backup solution. When Sergio got hold of the expert, the news was not surprising.

"Yeah, Peter told me about it this morning. I forgot what I did there in the first place. But once the system was installed, I haven't touched it since."

After getting the details, as paltry as they were, Sergio was able to confirm his expectation. The most recent backup was from 4 years ago. The expert had failed to enable the rotation option on the hardware, so as soon as the disk space was used up, the backups stopped. Supposedly emails were sent to the interested parties, but they seem to have been successfully ignored.

It was time to bring Peter back to reality. "Peter, the situation is not good at all. Your data is gone and your most recent back up is 4 years old. I've checked the site logs and while your software is way out of date, there's nothing to indicate a security breach on your web site."

You could almost see the light bulb above Sergio's head go on. Of course, the darkness of the early morning hadn't yet been dissipated, so that might have something to do with it.

The Logs. Back when Sergio was working for Communibox, he was still in his "let's log every single trivial action" stage of his career. The details were stored in a couple of database tables, ready for access when required. Now seemed like a good time.

"Hold on a couple of seconds", Sergio said to Peter.

After a couple of quick queries (well...the queries were quick to write. The results took a lot longer to become visible), an odd pattern emerged.

"Peter," said Sergio, "I'm looking at some custom logs. It looks like starting late on Friday night, someone went in and started deleting the product listings. Through the user interface. One at a time, it seems. For almost 40 straight hours. And the username is 'michelle'. Do who know who 'michelle' is?"

"What???", exclaimed Peter. "That's not possible. She was let go last Friday."

"And did you disable her user account?"

The silence was damning.

"I'll call you back."

Of course, Peter never did call back. A few months later, Sergio ran across Larry, a colleague who worked with him on the Communibox project. Naturally the late night data disaster came up in the conversation.

"Don't you remember Michelle?", said Larry. "You know, the brunette in the sales department with legs to die for and a smile that you couldn't resist? Well it turns out that Peter couldn't resist either. They were having an affair."

"That's not that big a deal. I mean it could be, but unless it got in the way of business, what's the problem?", said Sergio.

"Well, it's a problem if the CFO finds out. And the CFO is the CEO's wife."

Turns out that immediately after the CFO discovering the affair, Michelle was let go. Probably not the smartest decision, given the opening for a lawsuit that provided to Michelle. Of course, Peter was a little too busy with other things to even think about disabling an account. In fact, no one found out about the termination until the following Monday. And Michelle decided to take a different path than the legal one anyway. Apparently, the entire company was enlisted, full-time for three weeks, to do nothing be entering the information back into the system. Not an orthodox recovery plan, to be certain. But less painful (and less expensive) than Peter's upcoming divorce proceedings promised to be.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!