Fun Bugs: Spamming Jeff Bezos

From late 1999 through mid-2002, Amazon loved to send email that said "As someone who's purchased books by Joe Blow in the past, we'd like to offer you a 10% discount on his new book, 'My Life as a Joe'." The mail was powered by my company, E.piphany, and throughout most of 2000, I was at Amazon helping implement new features of that software. It was during that stint that I saw one of my Favorite Bugs of All Time.

One random afternoon, we were all happily hacking away, optimizing SQL queries, configuring new reports, when the VP of Customer Relations marched into the hallway and shouted, literally, shouted, "I NEED TO STOP ALL OUTGOING EMAIL RIGHT NOW. WHO CAN MAKE THAT HAPPEN?"

I was working with one other guy, and we immediately looked at each other and froze. We both knew that Amazon was very serious about protecting customers from spam -- by their rules, they could only mail unsolicited offers to each customer once a month. Two weeks earlier, our software had, by virtue of a bug, mailed a small set of customers more than once in the same month.

Now, a Vice President was standing outside our office, shouting at the top of his lungs, that he wanted to stop outgoing email. We were the most likely culprit, and we knew it.

We were in trouble.

It quickly came out that Jeff Bezos had received a few dozen email campaigns in the past half hour. Seeing these, Jeff concluded that Amazon's outgoing email had gone completely haywire, and we were spamming every Amazon customer around the world.

The rest of the afternoon was simultaneously gruesome and amusing. It slowly became public that someone at Amazon was working on a email marketing system to replace our software, and he'd been debugging it that day. Part of his code involved looking up a customer's email address, and the developer confessed that he had a couple means for looking up the address, and the fallback choice was "jeff@amazon.com". I imagine it looked something like this:

string GetEmailAddress(userid) {
  string result = GetEmailAddressMethod1(userid);
  if (result == "") result = GetEmailAddressMethod2(userid);
  if (result == "") // this can never happen
    result = "jeff@amazon.com";
  return result;
}

He was testing that code with his own ID, but he wasn't getting mail, so he kept testing it.

And testing it.

And testing it.

And every time he did, Jeff Bezos got mail.

For me, the bottom line was that we weren't responsible for the bug. Phew. The culprit and his boss had to explain themselves to Jeff, and it probably didn't help his career any, but nobody lost their job.

I'd love to be able to say there's some big lesson about software development to be had here, but the best lesson I can offer is "don't be dumb." I share this only for the entertainment value.