Wednesday, October 5, 2011

Email Bounce Processing with SaaSbouncer

One of my projects was sending a newsletter to its 270,000 member database for the first time, and we decided to use Amazon's Simple Email Service (the basics of that are in this blog post here).  Due to the age of some of the addresses in our membership database we were generating a large number of bounce backs and non delivery reports and we needed a tool to filter our bounces and remove them from our database.  We did some research and found bouncely which seemed like it might work, but I figured I could write a similar tool on my own. And SaaSbouncer was born.

SaaSbouncer was designed to accept all of your bounces and NDR's, process them, and give you access to the data via the API's

Step 1
First, you need to sign up  for the service which is quite simple, simply go to www.saasbouncer.com and register.  You will receive a 30 day free trial (or 500 bounces) by default.  Once you are signed in you will be provided with an email address along the lines of bounceXXXXXXXX@bounces.saasbouncer.com which will be specific to your account.

Step 2
Next, if you are using Amazon's Simple Email Service (SES) you need to verify your saasbouncer bounce address or you will not be able to send emails with this return-path; if you are not using SES you do not need to worry about this.  As per the saasbouncer instructions you can do this with the following command, replacing the address with your custom address:
ses-verify-address.pl -k aws-credentials -v bounceXXXXXXXX@bounces.saasbouncer.com


The verification email will show up in your non bounced messages where you can open it and approve it, or alternatively saasbouncer will generally approve it within a couple hours.


Step 3
The next step is to send your email.  If you are sending via Amazon's SDK for PHP you would do the following to set the returnpath to your saasbouncer account:
$m->setReturnPath('bounceXXXXXXXX@bounces.saasbouncer.com')


If you are using Amazon's SES you have one more advantage: the messageId.  The result of the sendEmail command returns an array, one of which is the messageId.  If you store this messageId in your database along with the email address you sent the message you will get near 100% accuracy with SaaSbouncer.  When saasbouncer receives a message we do our best to try and find the original email address the message was sent to but sometimes this is not possible due to email forwarding or masquerading.  However, the messageId will be available so in the API along with the email address the bounce was received for.  If you match based on messageId you will have near perfect accuracy.


If you are sending directly from PHP your message send function could look something like this, the important part is you set the return-path header:

<?PHP
$headers = "From: " . $from_name . "<" . $from_address . ">\n";
$headers .= "Reply-To: <" . $from_address . ">\n";
$headers .= "X-Sender: " . $from_name . "<" . $from_address . ">\n";
$headers .= "X-Mailer: PHP/". phpversion() . "\n"; //mailer
$headers .= "Return-Path: <bounceXXXXXXXX@bounces.saasbouncer.com>\n";
mail($email, $subject, $message, $headers);
?>


Unfortunately we do not have a unique messageId available to us when an email is sent in this format so we will not be able to track bounces based on messageId, only based on email address.

Step 4
Once you have sent your email you will want to retrieve the information about bounces.  It can take several days for all of the bounces and complaints to come in, but 90% of the bounces will come in within minutes of sending your outbound emails.  There are a couple ways you can get access to the bounce data. 

The most interesting option is the push notification API which sends an HTTP GET request to the URL of your choice.  You can configure and enable this option under account settings, and then the GET request would look something like this: http://yourdomain.com/saasbouncerendpoint.php?email=user@domain.com&messageid=0912512-12512-sadf-124as-12&maileremail=daemon@domain.com&datereceived=2011-12-15%2012:15:00&smtpstatuscode=5.0.0&emailstatus=failed&subject=delivery%20failure


Alternatively from the website you can generate a report using an interface that looks something like this.  You can select what data you would like in your CSV and do whatever you want with it.  

Finally, there is an API which allows you to download a CSV programatically, you use your account id and your apikey in the following URL and specify the day you want data for.  
http://www.saasbouncer.com/reports/csv.php?acctid=XXXXXXXX&apikey=XXXXXXXX&reportdate=2011-10-05

A good way to use this is to have a scheduled task that grabs this URL once a day (just after midnight), parses the CSV, and then marks the members as having invalid email addresses. 



<?PHP
$file = fopen("http://www.saasbouncer.com/reports/csv.php?acctid=XXXXXXXX&apikey=XXXXXXXX&reportdate=2011-10-05");
while($csvvalues = fgetcsv($file)){
//$csvvalues[0] = email
//$csvvalues[1] = messageid
//$csvvalues[2] = email subject
//$csvvalues[3] = email the bounce was received from
//$csvvalues[4] = date/time the bounce was received and processed
$email = mysql_escape_string($csvvalues[0]);
mysql_query("update member set invalidEmail = 1 where email = '$email'");
}
?>




No comments:

Post a Comment