'Maximum Number of Emails Per Hour Has Been Exceeded?' What the hell? The head of Golficionados was not pleased. He called James to get it fixed.
James and team had just launched the web site for Golficionados, a small golf supply business, the day before. As a part of the contract, James had to set up their email server and create accounts for all 30 users. He was surprised to be woken up at 6:30 by an angry client call.
He sleepily shambled to his computer and remoted in to the mail server. Sure enough, when he tried to send an email via the command line, he got a bounce back to his inbox with the same error his client was experiencing. Damn it, he thought as he looked up the number for the hosting company's support line.
While he was on hold, enjoying a muzak version of Journey's "Don't Stop Believin'," he tried to think of all the possible sources of the error. It could be a configuration issue, but he was ensured by the hosting company that they'd be set up for a reasonable amount of email, and certainly the first people in the office at 6:30 wouldn't have sent enough email to cause the error.
Finally, a Tier-1 support tech answered the call, sounding as tired as James. James explained the issue, and the tech said he'd check the logs. He was working for a while, interrupting the silence with the occasional "wait, that doesn't make sense..." or "hmm... why are the inbound messages being counted?" A few moments later, the tech exclaimed an Archimedean "Eureka!" He'd been transposing "incoming" and "outgoing" — the email was getting out and had been counted correctly.
"You've got a mail script or something that's been exploited," the tech insisted. "And spammers have taken advantage of the opening."
Ridiculous, James thought. He'd created a mail script himself, but it read the subject, body, from, and to addresses from a database — there was no potential to exploit the script without changing the code, and the emails were being sent from virtually every account on the server. James explained this to the tech, but he wouldn't budge on his diagnosis. "You're going to have to change your script. There's nothing we can do about this."
After suffering through the call, James had nothing to show for it. His phone started ringing off the hook from the client and other coworkers asking how to fix the issue, when it finally dawned on him — the "IT guy" at the client always advocated simplicity over security. "Let's give everyone the same password," he said, taking a page from the Tilde Financial Services book. "That way, if they have any problems, I can log in and help them." It was a self-fulfilling prophecy.
Yes, sharing a password is pretty terrible security. But as long as a strong password is chosen, at least some risk is mitigated. So what highly-secure, completely uncrackable password did Golficionados choose? "flog." "Golf" spelled backwards. But hey, it's backwards, thereby making it invulnerable to a dictionary atta- oh, wait.
With a password that could be cracked by someone who googled a simple dictionary attack script, someone broke in, hijacked the accounts, and started spamming. With the admin password (fortunately, one more complex than "flog"), James was able to stop the spamming for a little bit. He changed passwords as fast as he could type, ending the spam flood.
Moments later, the client called back, now with a new problem — no one could log in to their email. Rather than talk to the users directly, James sent an email to his boss. He provided instructions on how each user could select their own unique password, stressed the importance of proper security. "Fortunately, we've been through the worst that can happen, and now we know what to do so it won't happen again," he wrote. The mail server they used didn't have any options to enforce password complexity requirements, but the users could be trusted to follow written policy. Hopefully.
The following morning, James woke up to a 6:30 phone call again. "It says something about maximum emails per hour. I thought you fixed this!" The Golficionados lead was even more pissed off than before. James called support again. And listened to Journey again. And finally talked to the first-tier support guy again.
"So did you fix that script?"
"No, it wasn't that," James insisted. "But we're still getting the same error." After some convincing, James got the tech to look at the logs again. Again, "that doesn't make sense..." and "why is it counting these?" and finally, "Eureka!"
During the two hours that their server was sending spam, the email addresses on the server were signed up for every spam list imaginable. The accounts went from spammers to spamees, receiving thousands of messages which the mail server was counting toward the limit. The hosting company admitted that the software wasn't behaving as it should and was only supposed to have limits on outgoing messages, and assured James that the problem would be addressed immediately. Just overnight, though, the Golficionados lead had received over 2,000 spam messages to his Blackberry, depleting all the available memory and rendering it unusable.
The issue was fixed later that day and email was, for the most part, back to normal. Still, Golficionados thinks that everything was James's company's fault. The good news, however, is that once the mail was back to normal, the users were able to change their passwords back to something easy to remember. They all chose "flog."