Email Architecture

using Amazon SES

Deliverability and Scalability1

Introduction• Email is a method of exchanging

digital messages from an author to one or more recipients

• Email is one of the most critical component in an application for communication between the application and its users

• Sending effective emails help business to bring in more people to use their sites and to share information that helps the end users

2

How Email is sent?

• Internet was built on trust and email also works in the same fashion

• No authentication exists between ISP to send emails

• SPAM abuse is high as this trust is misused 3

What ISP’s are doing?• ISP’s nowadays actively

block IP ranges that they believe generate SPAM

• ISP’s employ a range of software to categorize email as JUNK, SPAM, Phishing, etc…

• Countries are actively legislating to punish offenders 4

What companies are worried about?

• Email deliverability• Ensure email reaches the

Inbox of the recipients

• Handle ISP reports• Complaints• Bounces

• Statistics

• Keep delivery costs low 5

Deliverability• Refers to the likelihood of the email reaching the intended

destination

• Majority of global email traffic is either unsolicited or undesired bulk email• 88 to 92% of email traffic in year 2010 were spam

• Due to the security measures of ISP, even genuine emails are marked as SPAM causing the company’s reputation to be downgraded from a users perspective

• Users are nowadays extremely cautious about clicking any links in email that are marked as SPAM 6

ISP reports• ISP’s respond to origin with reports on emails that were

processed by the systems

• Bounces – Messages sent to non existent email address

• Rejections – Size limitations and attachment types

• Complaints – Complaints about email content, etc…

• Email systems need to handle these responses and need to proactively handle the issue

• Ignoring reports can force an ISP to block the IP address of origin server causing email outage

7

Statistics• Email Statistics are an essential component of a business

application to understand the reach of emails

• Successful deliveries• Click tracking• Failure reports

• The above statistics are vital to understand the success of email marketing and campaigns

• This data adds business benefit to ensure that the customer can reach your system in case of emergency• Unable to deliver newly generated passwords since the users email

box is closed8

AMAZON SIMPLE EMAIL SERVICE (SES) 9

Amazon Web Services

Amazon Simple Email Service

Features• Simple Setup

• Zero Infrastructure

• Bulk & transactional email-sending service

• Standard SMTP service

• DKIM support

• API availability

Benefits• Simple

• Inexpensive

• Reliable

• Scalable

• No upfront costs

• Workflow Integration 10

About SES

What is it• Its an email delivery service

for bulk and transactional system

• Supports both SMTP and API based access

• Offers support for setting up DKIM and signing messages

• Offers endpoint to receive information on email delivery

What its not• Not a replacement for your

exchange servers. It doesn’t receive emails

• Not a marketing tool for sending bulk messages and track status. You’ve to build the system on top of SES

• Not a guaranteed email service that handles everything for you. It helps you to build a guaranteed delivery system.

11

How SES helps business?• Offers easy interface to help with complex setups like DKIM, etc..

• Offers a scalable, highly available service for sending emails

• Offers feedback loops to handle bounces and complaints

• Offers a no CAPEX model for your email delivery setup

• Automatically filters your content to prevent SPAM messages from being sent out if your infrastructure is compromised

• Requires no changes to existing applications since SES is compliant with SMTP protocol 12

DKIM Signing• DomainKeys Identified Mail (DKIM) is a standard that allows

senders to sign their email messages and ISPs to use those signatures to verify that those messages are legitimate and have not been modified by a third party in transit.

• DKIM signing helps in deliveryability of email messages as ISP’s can verify the sender’s domain for authenticity.

• DKIM Setup in SES can be done in 2 ways• Easy DKIM using the SES console

• Manual DKIM signing by using the SES REST API

13

Easy DKIM Signing• Verify a new Domain• Generate DKIM Settings• Add the DNS records with

your registrar• 1 TXT record• 3 CNAME records

14

1

2

3

Error Handling

• SES offers endpoints to handle the following errors:-• Bounces• Complaints

• SES can be configured to handle errors in the following ways:-• SNS topics (HTTP endpoints)• Email Forwarding

15

Send Rates• Depending on the needs of the application, SES throttles the

maximum messages that you can send in a second

• Adherence to this rate is essential to ensure SES service availability

• You can request the SES team to increase the rates if your application needs are more

• Send rates improve based on email quality and decreases based on JUNK/SPAM/Bounces

16

SCALABLE EMAIL ARCHITECTURE 17

Amazon Simple Email Service (SES)

Scalable Email Architecture

18

Delivery Workflow• Application logs a queue message for sending email

• Email service polls the queue to get the message

• Locks the queue message as being processed

• Parses the message to identify the message and recipient

• Calls SES to send the email message to the recipient

• Deletes the message from queue on success

• Marks the message as unread if failure in SMTP connectivity19

Email Service• Independent Tier to handle Email delivery

• Can be multi threaded / multi instance for scale

• Distributed architecture for handling failures

• Has logic to handle the following• Log bounces to prevent resending to an non existent address• Control send rates based on configuration• Failover in case SES is down to switch to a different SMTP service

• Can be independently tested

• Can offload logic to generate email messages based on a email event – template pattern

20

Bounce Handling• SES receives the bounce message from ISP

• SES sends the message to the SNS topic configured for bounces

• SNS sends the message to the subscriptions• Can be an email address to manually handle the bounces• Can be an HTTP endpoint for an application to process the message immediately• Can be an SQS queue for distributed processing (Current Architecture)

• Email Service polls the SQS bounce queue to receive the bounces

• Takes appropriate actions based on the setup• Mark the email address as non functional in database• Maintain its own store of failed email address (Validates against this every time

when an email needs to be sent out)• Sends an alert to the user’s alternate email address about the applications

inability to reach the user

21

Load Testing your service• The email service can be independently tested as it is not directly

linked to the main application

• Email service capabilities like bounce handling can be tested by using the new SES mailbox simulator

• You can read more about the mailbox simulator here

• http://aws.amazon.com/about-aws/whats-new/2012/10/03/amazon-simple-email-service-announces-mailbox-simulator/

• Using the mailbox simulator will not affect your SES reputation 22

SES safety measures• SES is a cost-effective email delivery service

• Being cost-effective makes it attractive for spammers

• Bounces are unavoidable in an internet accessible application that allows registrations.

• SES evaluates your reputation on the following basis:-• Bounce Percentage• Complaints received• Quality of emails

• SES takes these parameters seriously and can block production access when SES service is abused

• As long as the bounce percentage and the quality of emails are good, SES is a cost-effective service for your emailing needs

23

HAPPY EMAILING 24

Jithendra Balakrishnan

All Images copyrighted to respective authors