Mike's Page

Yesterday I sent a monthly newsletter mailing for one of my clients. Here are some numbers for fun:

• 13,037 recipients
• 208 bounces of various kinds, 1.6% (which I will handle using the technique I described last week)
• 2,945 AOL users, 22.6%
• 12 AOL users marked the message as spam, of those:
  • 11 signed up at the website in the last year
  • 2 signed up at the website in the last month
  • 1 had bought items from the site

I got the AOL spam reports by setting up a "feedback loop" where they inform you when someone complains about a message being spam. It seems reasonable to do in the case of AOL, since a large portion of the site's subscriber base uses them, but I can't imagine having to set up and monitor this stuff for every ISP. Currently I'm manually looking at those spam reports, but I may have to automate it eventually.

I suppose some of those unhappy AOLers could've been signed up for the list by someone else, but the site I do this for is not the kind of place you'd expect to see malicious users. I'm guessing that some of them marked it spam out of laziness instead of replying to unsubscribe, or using the website's email options. Also, according to this Slashdot article, the latest version of AOL makes it too easy to accidentally mark something as spam.

I don't want to be sending mail to anyone who doesn't have an interest in it. It's simply more work for me.

I do work for some websites that send email to customers. Order receipts, customer support, periodic newsletters, etc. I became an email administrator without knowing it. Now I have to find ways to automatically deal with email that bounces. That can happen for various reasons, like a user having an invalid address, being over quota, or blocking a message with a whitelist/overly-agressive spam filter (my clients aren't spammers).

I have things set up to bounce most errors to a special bounce email account. You can do this by adding an Errors-To header in your messages. This way I have them all in one nice email account, and I can try to process the errors automatically. I set that up a couple years ago, and I've since found out that there are tons of different ways for mail servers to send you errors.

My first attempt to deal with the messages was messy. I spent time looking at the various errors and came up with some common ground between them, like subject lines and the message bodies. I grabbed a PHP POP3 class which allows PHP to read and handle mailbox messages. Then I wrote regular expressions to try to find the email address in question, and then unsubscribe it from the mailing list. This processed a lot of messages but never handled them all...too many variations of error messages.

My latest method is to use something known as plussed email addresses. Some mail servers recognize an email address like user+data@domain.net as email for user@domain.net. Our server (sendmail) allows this. An application can use the "data" part to do something useful. Now I set the Errors-To to be something like bounce+userid@domain.net. I was able to ditch all the regular expression subject parsing and simply lookup the user id from the plussed data and unsubscribe them. This works great. (One note: I had to also set the Return-Path header to the same address to get this to work...not sure if some mail servers don't like that + sign when it comes to bouncing messages, though it seems to work fine now)

Obviously I'm not handling the type of bounce error. So I'm probably unsubscribing some users who were just temporarily over their email quota or something. That means I have room for improvement. I'm not going to go back to full-scale regex parsing to find the error codes and stuff. Many bounces come with an attachment called "message/delivery-status" which often contains a standard format (I should probably have been using that file when I wrote the first implementation). My eventual plan is to use the error data in that file and implement some kind of scoring system which will allow for a few minor errors before I completely unsubscribe a user.

There are probably better ways out there to handle this stuff (email admins feel free to comment!), but I haven't seen them, especially in PHP. I did come across a Perl CPAN module for handling bounces, but everything's in PHP right now...I'd prefer to keep it that way.

If only I lived in the San Francisco Bay Area, I could combine two of my favorite sports!

(By the way, I don't think the guy mentioned is the Oracle billionaire.)

We met Liza and Jim on our 2002 Everglades Trip, sharing a camping platform with them in the backcountry. Liza has a collection of wildlife art on her website. We recently ordered a couple prints: Light in the Maples and Forest Floor May. They turned out beautifully and were professionally framed and carefully shipped.