1

Fighting Spam at AOL: Lessons Learned and Issues Raised

Carl Hutzler

Director of Anti-Spam Operations

America Online, Inc.

12/9/2005

2

Agenda

• Email Identity Technologies

• Email Forwarding

• Email Service Provider Best Practices

3

What do Email Identity Technologies Do?

• They provide some assurances that a domain is being used with permission– Citibank can control the use of their domain, but cit1bank.com will

still be abused– Bounces can be analyzed to see if they are legitimate– Information can be analyzed on the responsible domain owners and

their reputation/accreditation

• But remember, email identity technologies do not stop spammers!– They only force spammers into other behaviors, many of which are

better for enforcement and controls.– But without message providers doing their part to use these

technologies wisely, we will be no better off.

4

AOL is a Crystal Ball

• Bulk Mailers on AOL’s whitelist comprise 30-50% of our daily email volume but only 5-10% of complaints.

• >80% of AOL’s spam problem comes from other provider’s main outbound MTAs and compromised web servers (CGI scripts)

• AOL began seeing this shift in Sept 2003

• The rest of the internet is beginning to see this now…

– “We're the biggest spammer on the Internet," network engineer Sean Lutner, Comcast - source CNET.com, May 24, 2004

Report from 9/14/2004

188841 hotmail.com64543 x-mailer.co.uk62757 shawcable.com46312 concentric.net32259 cnchost.com32022 zero.ou.edu23557 mail.atl.earthlink.net22837 grp.scd.yahoo.com21005 ucla.edu17676 oemgrp.com16849 mail.cornell.edu16260 dejazzd.com15764 mta01.tie.cl15659 mrf.mail.rcn.net14343 urbanhomesecurity.com14280 mail.pas.earthlink.net14246 smtp.nextra.cz13646 mail.yahoo.com

Note1: Greyed domains have very low spam penetration due to very large number of emails sent which counters the total complaint statistic.

Note2: Italic domains were whitelisted and subsequently blocked for spamming.

5

All spam will eventually come from Email Message Provider Networks

Hacker/Spammer

MyDoom’d ZOMBIE PC on DSL.NET

mx.aol.com

BLOCK outbound1.dsl.net

For example: AOL, BlackLists, and other organizations are getting really fast at blocking zombie machines

BUT…

The machines do not get un-infectedNo SMTP AUTH

• Most ISPs “trust” internal networks

No Outbound Spam controlsNo Rate controls

Results? ISP mail servers act as forwarding MTAs for a network of open relay Zombie machines

6

Will SenderID, SPF, DomainKeys, etc stop spam?

• Simple answer, NO. Complex answer, NO.• Why?

– Most AOL spam obeys sender identity technologies TODAY!– Spammers send through the local MTA and use the local ISP’s domain as

the FROM/Sender

• Identity Technologies can allow blacklists/whitelists to work from DOMAINs instead of IP addresses

– Good from a not blocking innocents by IP address standpoint– Reputation/Accreditation systems will be key to success of Email Identity

technologies– Without SMTP Authentication, we are only validating the DOMAIN and not

the USER portion of the address (user@domain.com)

Bottom Line: If ISPs don’t get smart soon and control the sources of spam on their networks, the reputation for their domain (e.g., comcast.net) will be so poor that they

will not have connectivity to other ISPs

7

Email Forwarding

8

Forwarding Spam to AOL Customers

• AOL can only trust the IP address of the client MTA that connects to an AOL server– No other headers can be trusted as they are all forgeable– This is why internet whitelist/blacklists are all done by IP

address.

• AOL has no way to no that a message is simply a forwarded email– Does this even matter?

9

So what happens when a University FORWARDS Spam?

• Generally, if AOL gets enough complaints from our members, we block or temp fail the IP address

• Is this the members fault?– No, as there is nothing in the email that shows it is from their

forwarded account– AOL members do not read headers, nor should they be

expected to.

10

Possible Solutions?

• Dedicate an IP address to handle forwarded mail and tell AOL about it.

• Do better spam filtering inbound to your network.• Spam filter the outbound traffic and insert a spamassassin x-

header that identifies a message as spam. AOL will spam folder it.

• Change the headers of forwarded mail to identify the situation to final recipient.– From: ForwardedEmail@university.com

– Subject: [FORWARD] Original Subject

– ReplyTo: originalsender@originaldomain.com

Bottom Line: Forwarding spam to someone’s inbox innocently or intentionally still creates a bad experience for the final recipient. Port25 is your responsibility.

Bottom Line: Forwarding spam to someone’s inbox innocently or intentionally still creates a bad experience for the final recipient. Port25 is your responsibility.

11

Mail Service Provider Best Practices

12

Message Provider Code of Conduct:Take Responsibility for outbound Port 25

• ISPs must take full responsibility for all traffic/messages emanating from their network on port25.– Port25 traffic is always Unauthenticated traffic and as such

must be accepted by server MTAs.– Abuse issues are always the responsibility of the

sending/client MTA

13

How does a Message Provider like AOL control outbound port25 traffic?

• Hijack all direct port25 connections from dynamic IP space to other ISP mail servers and process it for viruses/spam.– Other providers block port 25– Still others use a mail proxy to detect SMTP authentication

credentials and only allow authenticated SMTP traffic on port25– Some simply rate limit how much a single IP can send if their IP

space is rather static or they can tie an IP to a customer account

• Rate limit all customers through outbound, authenticated MTAs. Rate limits per hour and per day work well.

• Monitor complaints about customers via the SCOMP Feedback Loop system

• URL blocking for known spammer URLs• Secure accounts that are spamming - thousands daily

14

Summary: What technologies will stop spam?

• ISPs and Network Providers “waking up” and working together to cut off the spammer’s oxygen supply:

– Spammers need connectivity– Spammers need large numbers of high throughput IP addresses

• So what is the formula for success?– ISPs should monitor their networks for sources of spam LEAVING their

network• Port25 is always the responsibility of the originating ISP• Shift some of the resources from inbound filtering to OUTBOUND Controls

– Enforce strong authentication to authorize use of an ISPs MTAs– Monitor customer sending patterns like a credit company monitors

“fraudulent charges”– Monitor/Sign-up to receive complaints from AOL and other sources

(spamcop, abuse@, etc)– Remove sources of spam within minutes (Zombie machines, insecure CGI

scripts, bad customers, etc)

15

Thank you!

• For more information, contact Carl Hutzler:– cdhutzler@aol.com

• Delivery issues to AOL?– See if your network is a source of spam

• http://postmaster.aol.com/

• Click on the “Feedback Loop” Button

– Contact the AOL Postmaster 24x7• 1.888.212.5537