I may have ignored your emails, unintentionally

October 22, 2009

Apologies to anyone who hasn't received a response from me to an email sent within the last ~month. I realized today that many of my incoming emails were being picked up and held for questioning along the route to their final destination. GAH! (If I don't know you personally, the fact that I haven't yet replied doesn't necessarily mean anything; see the second paragraph of this post.)

Many times over the course of the last few weeks I've not received an email when I was expecting one (e.g., in response to an email I sent, an ongoing conversation, an order acknowledgment, etc.), which has led to a number of awkward situations and broken conversations (that now make perfect sense). The oddest part about all of it was that it was so seemingly random. For example, active threads would just stop updating (from my perspective) for a day or two until a third person on the chain would reply, at which point I'd see portions of the conversation I'd missed, and then emails belonging to those who were briefly "blocked" would start coming through again. Also, emails from any account I had control over (e.g., work, old school accounts, etc.) would come through every time.

Finally, today, after not receiving an expected email, and after having the person try to send it multiple times from two separate accounts, I knew I had to figure out the problem and fix it. Though the issue turned out to be rather uncomplicated (and easy to resolve), it wasn't immediately obvious to me, especially after six months of problem-free email service.

When I moved this site to Slicehost early this year, I decided I didn't want to fuss with setting up email daemons on my (virtual) server, and so I outsourced the entire operation to Google Apps, which, after you modify your domain's MX records to point to Google, takes care of everything.

Instead of actually using my justin@justinblanton.com account (now handled by Google Apps), I decided to have it forward all of my email to my jblanton@gmail.com account, mainly because all of my email from the past few years already is there (I've been forwarding to it for a while) and I just didn't want to go through the hassle of migrating all of it over to my Apps account. Further, the jblanton@gmail.com account already knew my preferences, which contacts I interacted with most, what I thought was and wasn't spam, etc.

After setting up the forwarding rule on the Apps account I never touched it again. For seven months. Until today.

In the wake of ruling out every conceivable cause of the missing-email issue, it finally dawned on me that maybe emails were being flagged as spam before they got to my Gmail account. Sure enough, I logged into my Apps account and had 120,000+ marked-as-spam emails waiting for me (all from just the past 30 days), many of which were the licit emails I was missing. Again, I experienced no email issues until a few weeks ago, and so I'm convinced that a ~month ago something changed on Google's backend that caused its spam filter to become a bit overzealous.

I searched the spam folder for individual names and recovered what I could, but obviously I couldn't search for everyone in my contact list, and who knows what number of emails I'm missing from people who emailed me for the first time (hopefully they're subscribed to the site and are reading this now). Also, I'm able to go back only a month because Google deletes messages flagged as spam after it's held onto them for 30 days. Like I said though, I think this issue crept up just a few weeks ago, and so I likely won't miss any false positives (that I actually search for).

After recovering what I could, I created a forwarding filter using is:unread in the "has the words" box, and then had that act on every legit email in the inbox. Unfortunately though, Gmail/Apps won't let you forward en masse emails you've already received. Thus, if I want to get these recovered emails over to the jblanton@gmail.com account, I'm going to have to forward each of them individually.

I've just transferred my contact list from the Gmail account to the Apps account, which should help to stop the bleeding going forward (i.e., at least those in my address book won't be marked as spam). I likely will jump into the Apps account a few times over the next couple of weeks in an effort to train the filter. To that end, is anyone aware of a way to 1) have a Gmail/Apps account forward all email (including what it flags as spam) to the address you provide; or 2) migrate, between two Gmail/Apps accounts, all of the is-spam/is-not-spam knowledge you've imparted to it over the years?

Anyway, this obviously was just a terribly long-winded way of saying I'm not ignoring you, or maybe I am, but at least I've a great excuse if you ask!  ;)

You should follow me on Twitter here