What You Need to Know About Email Authentication

What You Should Know About

EmailAuthentication

Kurt Andersen

LinkedIn Site Reliability

@DrKurtA

@DrKurtA

Introduction - Kurt Andersen Email in the days of Bitnet/DECnet/usenet/etc. gateways

Early work on hypertext for internal reference system at JPL’s Image Processing Laboratory

Instigator for the PERL common database access framework: DBI/DBD

Early web work, bringing several organizations on line in 1994-1996

Worked on early versions of SpamAssassin, contributed to Postfix and the initial SPF specification

Active contributor to M3AAWG since 2007, currently on the board of directors and co-chair of the program committee

Active with IETF standards efforts: SPFbis, DMARC, DBOUND

@DrKurtA

Audience Check

What brings you to this session?

@DrKurtA

Roadmap

Why should you care?

Background to the challenges of email authentication

Where does me.example mail come from?

Did my domain (me.example) send this content?

What protections should a receiver look for on me.example mail?

How do I protect me.example?

Making it all work together. . .

@DrKurtA

Just

Ho

w B

ig Is

Th

is T

hin

g?

2

This is really big.

Lots of people have been trying to fix

this thing for a long time.

..and it’s actually changing!

@DrKurtA

Why Is This Important? Email is still alive and well

- Large: 80B consumer emails/day, 91% check email daily

- Growing: 3.9 Billion active email boxes => 4.9 Bn by 2017

- Preferred: 74% consumers prefer email for commercial communications

- Popular: 82% of consumers sign up for email programs on websites

Email is highly effective

- High open rates: 82% of consumers open marketing email

- Effective: 66% of consumers buy online due to email

- Efficient: Email marketing has an ROI of 4300%

@DrKurtA

Email Attacks

84% of all email is spam/phish

Despite best efforts, 100M Phishing messages get through every day

- 95% of all cyberattacks occur through phishing

Phishing harms consumers and brands

- Daily barrage of email based attacks costing brands $70 b/year

- 42% of consumers are less likely to buy from brands following an email attack

@DrKurtA

Domain Authentication Helps Receivers

It makes it easier to know where the mail is really coming from

Stable basis for accumulating reputation

- The only people who don’t want to accrue reputation are the people you don’t want email from anyway

Necessary to deal with the shift from IPv4 to IPv6

@DrKurtA

Domain Authentication Helps Senders

It keeps your mail out of the trash (assuming that it is good )

Keeps your real mail distinct from fraudulent uses of your domain

Consider it the domain equivalent of defending your brand

@DrKurtA

A Tale of Two Cities Email standards – IETF RFCs govern how servers talk to each other

- 5321 – envelope

- 5322 – message (and others for MIME, etc.)

- Various RFCs for authentication protocols

- MTA “rules of the road”

MUA – how are messages presented to people

- No standards

- Some user interface optimizations

- Little real research toward usable security

@DrKurtA

Background and Context








@DrKurtA

Challenges of Email Authentication

There’s a reason that SMTP starts with the word:

Simple

@DrKurtA

Message Sender Message Receiver Notes

TCP connect on port 25 Standard 3-way handshake

220 testhost.localdomain ESMTP Postfix

Connection banner

EHLO test.example.com 250-testhost.localdomain250-SIZE 10485760250250 DSN

Receiver announces capabilities (EHLO) or just “OK” (HELO)

MAIL FROM: <[email protected]>

250 2.1.0 Ok Envelope sender

RCPT TO: <[email protected]>

250 2.1.5 Ok Envelope recipient

DATA 354 End data with <CR><LF>.<CR><LF>

<message with headers goes here> Message

. 250 2.0.0 Ok: queued as 0FC77B8BEDC

End of message

QUIT 221 2.0.0 Bye

Connection closed receiver

@DrKurtA

Designed by Analogy to Physical Letters

But who is it “from”?

@DrKurtA

Email: Envelope + Headers + Content

@DrKurtA

Connecting the AnalogySnail Mail Email Ifs, Ands & Buts

Postmark • IP address of connection• PTR record (sometimes called rDNS)

• HELO name

• Cost of admission• Frequently required

• sometimes with FCrDNS too• Often unrelated to anything

Envelope From aka “Mail From”/ “MFrom” / “5321.From” • Becomes the “Return-Path” header

• Named from RFC5321 which defines the envelope for email

• Usually null for system messages

Envelope To aka “Rcpt To” / “Recipient” / “5321.To” Used for delivery, but not seen by recipient

Letter From aka “Header From” / “HFrom” / “5322.From” • Display Name – seen, free text• Email Address – often unseen

Letter To aka “Header To” / “Recipient” / “5322.To” optional

Signature none

@DrKurtA

First Axiom of Email Reputation

You cannot say good things about yourself,

only neutral or bad things.

– John Levine

@DrKurtA

SPF








@DrKurtA

Where does “me.example” mail come from?

Sender Policy Framework, aka, SPF RFC 7208

ADMD (ADministrative Management Domain) assertion about the source(s) of email for a domain

Published in DNS as a TXT record

@DrKurtA

What does SPF check?Snail Mail Email Ifs, Ands & Buts


• HELO name









Signature none

@DrKurtA

Particulars of an SPF record v=spf1 {list of qualifiers + mechanisms}

Qualifiers:

Mechanisms:

+ PASS (default)

– HARD FAIL

? NEUTRAL ~ SOFT FAIL

ip4 ip6 a

mx all exists

include redirect ptr (deprecated)

@DrKurtA

SPF Macros

Allows mechanism definition based on IP or various parts of the 5321.From (MFrom) address

Not all receivers support all macros which are defined in the spec

Some receivers ignore macro-laden expressions

Use with care

@DrKurtA

SPF Record Example for “me.example”

v=spf1 ip4:1.2.3.4 ip4:6.7.8.0/24 a ~all

@DrKurtA

Evaluating SPF Starting with the domain of the 5321.From (MFrom), look up the TXT record for that

domain to find the SPF record

- If 5321.From is empty, use the domain listed in the HELO/EHLO identity

If there is no record, the result is NEUTRAL

If there is an SPF record, test each mechanism from left to right stopping after a match is found

- The qualifier determines the result of the match

- Most SPF records end with “–all” or “~all” to provide an unambiguous result

@DrKurtA

SPF Record Example for “me.example”

v=spf1 ip4:1.2.3.4 ip4:6.7.8.0/24 a ~all

@DrKurtA

SPF and Third Parties A domain owner can authorize third party sources with “include”

But…- SPF does not support intermediaries in the delivery process

- Intermediaries account for ~5% of email received at several large mailbox providers

@DrKurtA

SPF Gotchas DNS lookups, including embedded “include:” are limited to 10 queries

- Put non-DNS mechanisms first and put highest volume sources first

Use “~all” instead of “–all”, because some people on the internet will drop email if SPF fails and “–all” is in place.

Record length can oddly matter. Try to fit SPF into a UDP packet (~500 bytes).

@DrKurtA

More SPF Gotchas

DNS time-to-live (TTL) will affect how quickly changes can be made

Publish SPF records for sub-domains (or use wildcards)

- SPF does not “discover” SPF records if they’re not present

Use tools to check your SPF record

- Tools separate humans from most other creatures.

- Safe to say: Smart creatures use tools.

Be careful what you “include” – other records may be broken or wrong

@DrKurtA

SPF Q&A








@DrKurtA

DKIM . . .








@DrKurtA

What is DKIM and what does it do? DKIM == Domain Keys Internet Mail

Public key crypto to sign a message: body + selected headers by an ADMD

Private key held by the ADMD

Public key published in DNS by the ADMD based on a “selector”

@DrKurtA

Sample DKIM Signature

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoogroups.com; s=echoe; t=1393079384; bh=kmukFXBXZ2LCalggiEXX2pc4h9ESv+STtGxZ/NFuN+k=; h=Received: Received:X-Yahoo-Newman-Id:X-Sender:X-Apparently-To: X-Received:X- Received:X-Received:X-Received:X-Received: X-Received:X-Received:X- Received:X-YMail-OSG:X-Received: X-Rocket-MIMEInfo:X-Mailer:Message- ID:To:X-Originating-IP: X-eGroups-Msg-Info:From:X-Yahoo-Profile: Sender:MIME-Version: Mailing-List:Delivered-To:List-Id:Precedence: List-Unsubscribe:Date:Subject:Reply-To: X-Yahoo-Newman-Property: Content-Type; b=5KWzHV7YzWaUURDQW/MKelqHkdy8V/ube+c2P8+c4yX+CFKHPsk9j76G 3Yt25L7DQLU3djFacfVbdZdxz/Y41TmNcq4FVXZ23ZC42m9Ku6AN3uSxLG Jm9KbrQ5/P2+pvaJHC NwecnPm1P+EiYu3qsY1FCywYTJ4GxGpkqBKRFfg=

@DrKurtA

Finding the DNS record for DKIM Look for a TXT record at <s=>._domainkey.<d=>

$dig txt echoe._domainkey.yahoogroups.com +short

"k=rsa\;p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDmsJgfzmZfV10FE4jZ9NAX62SchSffsRHR/ng8TfS8YT33pdMMcUgthGXCw+n7xZOYyYvbII2OemMv0quJLUZfJFfJj2QSwI49qO3K04cUv0pNFt3/ugWzKl65Hgx1pLAoux5hdtJAmUJKM+kaaLaG6nR/qJT2iALWAGqoB2UhOQIDAQAB"

@DrKurtA

What does DKIM check?Snail Mail Email Ifs, Ands & Buts


• HELO name









Signature none DKIM – sort of

@DrKurtA

DKIM and Third Parties A domain owner can authorize third party sources several ways

- providing private keys to trusted third parties

- publishing the public keys from trusted third parties

- delegating (sub)domains to trusted third parties

- either full delegation or

- via CNAME or DNAME assignment mechanisms

But…- Don’t use the same key across all of your subdomains

@DrKurtA

DKIM Gotchas Weak Keys: 1024 bits is the minimum acceptable strength at this time

Typos in DNS records

DKIM does not work everywhere

- Implementation bugs

- Gateways that break the signatures

@DrKurtA

DKIM Best Practices Key rotation: Essential to good security

- Rotate all keys at least twice a year

Automate Everything!

- Far less chance of errors

- “Push a button” to rotate keys if keys get compromised

- Automate everything includes:

- Key generation, publication, rotation – inhouse and outsourced

@DrKurtA

DKIM Q&A








@DrKurtA

DMARC . . .








@DrKurtA

What is DMARC? DMARC = Domain-based Message Authentication, Reporting, and

Conformance

- Authentication – Leverage existing technology (DKIM and SPF)

- Reporting – Gain visibility with aggregate and per-failure reports

- Conformance – Standardize identifiers, provide flexible policy actions

RFC7489 (Informational)

@DrKurtA

How DMARC works. . . Starting from the 5322.From (HFrom) domain

- look up the TXT record for _dmarc.<domain>

- if that does not exist, lookup _dmarc.<org domain>

Check for authentication success + alignment (HFrom domain):

A. If the SPF result was “PASS” and the SPF domain matches

B. If any of the DKIM signatures validate and the DKIM domain matches

If (A or B), then DMARC PASS, else DMARC FAIL and the resulting action is based on the DMARC record

@DrKurtA

Basics of the DMARC record

$ dig txt _dmarc.yahoogroups.com +short

"v=DMARC1\; p=none\; pct=100\; rua=mailto:[email protected]\;”

$ dig txt _dmarc.paypal.com +short

"v=DMARC1\; p=reject\; rua=mailto:[email protected]\; ruf=mailto:[email protected],mailto:[email protected]

v version

p policy

sp subdomain policy

pct apply to X% of mail (defaults to 100%)

rua Reporting URI for Aggregate reports

ruf Reporting URI for Failure reports

Lesser use:adkim, aspf alignment policy for DKIM, SPF (defaults to “relaxed”)

@DrKurtA

What is an “Organizational Domain”?

For more information see:

PublicSuffix.org and

the IETF DBOUND working group

From Domain Organizational Domain

host.a.b.c.example.com example.com

ses.amazon.co.uk amazon.co.uk

a13-14.smtp-out.amazonses.com

amazonses.com

www.perkins.pvt.k12.ma.us perkins.pvt.k12.ma.us

@DrKurtA

What do the three DMARC policies mean?

none

quarantine

reject

local policy

@DrKurtA

What does DMARC verify?Snail Mail Email Ifs, Ands & Buts


• HELO name









Signature none DKIM domain

@DrKurtA

The Cousin Domain Problem

Look-alike or look-similar domains: bankofamerica.com vs. banckofamerica.comlinkedIn.com vs. linkedln.com (first one is linkedIn)

Unicode trickery: “Joe User” <jο[email protected]οm>

Puny-code trickery: alice@岍岊岊岅岉岎 .com (many MUAs will display [email protected])

DMARC only protects exact matches (or subdomains thereof)

@DrKurtA

DMARC Reporting – Aggregate Reports (rua)• Each report covers one 5322.From domain• You should get one from each supporting mailbox provider that sees email

with your From domain • Daily by default

XML Format• Organized by sending IP address (as seen by receiver!)• Contains

• Authentication Results (DKIM, SPF)• Alignment Results• Policy actions taken• Reasons for not taking policy actions

Just publish a “p=none” record to start receiving these

@DrKurtA

DMARC Spec – Reporting

XML Format

The policy that was found by the receiver:

<policy_published><domain>facebookmail.com</domain><adkim>r</adkim><aspf>r</aspf><p>reject</p><sp>none</sp><pct>100</pct>

</policy_published>

@DrKurtA

DMARC Reporting<record> <row> <source_ip>106.10.148.108</source_ip> <count>1</count> <policy_evaluated> <disposition>none</disposition> <dkim>pass</dkim> <spf>fail</spf> </policy_evaluated> </row> <identifiers> <header_from>facebookmail.com</header_from> </identifiers> <auth_results> <dkim> <domain>facebookmail.com</domain> <result>pass</result> </dkim> <spf> <domain>NULL</domain> <result>none</result> </spf> </auth_results></record>

XML Format

An example record

@DrKurtA

DMARC Reporting – Failure Reports (ruf)• One per DMARC failure

• AFRF or IODEF formats

• Should at least include ‘call-to-action’ URIs

• Throttling

• Privacy considerations• Might be redacted• May not be supported by all receivers that otherwise support DMARC

@DrKurtA

Making sense of DMARC reports

Do It Yourself

- Various open source tools and libraries are available

Outsource

- Various vendors are available

For more specifics, consult http://dmarc.org/resources

http://dmarc.org/resources

@DrKurtA

DMARC Gotchas

DMARC relies on SPF and DKIM – if those aren’t done right, DMARC will not work either

Separate your employee mail stream from your product mail stream (use different domains so you can have different policies)

If you delegate a domain, watch out for vendors over-riding your top-level policies

Expect to find mail streams that you did not know about!

@DrKurtA

DMARC Q&A








@DrKurtA

Protecting “me.example”. . .








@DrKurtA

Protecting Content From Prying (5)Eyes

Invest in properly configured TLS- Email is an interface to your company and product just as much as your

website

- Use certificates signed by reputable CAs or DANE, not self-signed ones

- Make sure your cipher suite list disallows weak and compromised ciphers

- Implement all of the appropriate security controls to prevent downgrade attacks

@DrKurtA

The Achilles Heel of the Internet DNS

Built on trust, just like SMTP was

In an untrustworthy world – security is critical

- Invest in understanding DNSSEC to make an informed decision for your domain(s)

- Invest in understanding DANE – an alternative to 3rd party CAs

http://www.internetsociety.org/deploy360/start/



@DrKurtA









@DrKurtA

Protecting Parked or No Email DomainsNo mail is sent from this domain

- SPF: v=spf1 -all

No mail is received by this domain- “Null” MX: “MX 0 .”

But tell me about any attempts to abuse this domain- DMARC: v=DMARC1; p=reject; [email protected]

Example: gmail.co (Columbian TLD mis-spelling for gmail.com):- v=spf1 -all

- v=DMARC1; p=reject; rua=mailto:[email protected]

@DrKurtA









@DrKurtA

Reporting and Compliance For Domain Owners

@DrKurtA

Initial Record Publishing

Everyone’s first DMARC record:

v=DMARC1; p=none; rua=mailto:[email protected];

mailto:[email protected]

@DrKurtA

3rd Party Deployment Profiles

Controlled – The Domain Owner fully controls their own DNS, and wants as much

control over their email as possible.

Authorized – The Domain Owner lets the 3rd party dictate the content of some DNS

records, while still retaining some operational control.

Delegated – The Domain Owner delegates control of some or all of their DNS to a 3rd

party, and wants to be mostly hands-off with their email.

Hosted – The Domain Owner allows the 3rd party to handle everything, and has little

control

@DrKurtA

3rd Party Deployment Profiles – Controlled

The Domain Owner retains control of the domain or subdomain, provides DKIM signing key(s) to 3rd party/ies and publishes the public key(s), and includes the appropriate information in their SPF record.

Pro• This scenario allows 3rd parties to send as the organizational domain if desired • The Domain Owner retains operational control

Cons• Coordination between the domain owner and the 3rd party mailer(s) is required to ensure

proper DKIM key rotation, accurate SPF records, etc. • Risk of coordination overhead/issues increases as the number of bilateral relationships

increase for domain owners and vendors.

@DrKurtA

Contractual points:

• Process for DKIM key rotation. Obligations of each party, including testing

• SPF record requirements and process for adding new hosts

3rd Party Deployment Profiles – Controlled

@DrKurtA

Similar to Controlled Profile, except the 3rd party creates the DKIM key pair(s) and generally takes a more active role in dictating record content. This approach is useful for Domain Owners where a different 3rd party is providing DNS and other services for the domain.

Pros• Can streamline provisioning for the 3rd party (but only the first one!)• One less task for the Domain Owner

Cons• Can create additional management issues for Domain Owners who use multiple 3rd

parties• Possible additional contractual point for key strength requirements

3rd Party Deployment Profiles – Authorized

@DrKurtA

The Domain Owner delegates a subdomain to 3rd party mailer and relies on contractual relationship to ensure appropriate SPF records, DKIM signing, and DMARC records

Pros• Reduces Domain Owner implementation issues to mostly contractual. • The 3rd party is responsible for SPF records, DKIM signing and publishing, etc. • Domain owner may still be responsible for ensuring Identifier Alignment.

Cons• The Domain Owner potentially gives up day to day control,

flexibility/responsiveness and visibility into operations and conformance.

3rd Party Deployment Profiles – Delegated

@DrKurtA

Contractual points

• Creation and maintenance of SPF, DKIM and DMARC records• (At least every 6 months) Rotation of DKIM keys and minimum length of key

(1024 or larger)

• Investigation of DMARC rejections• Handling of DMARC Reports• Requirements for reporting back to the Domain Owner• Indemnification (if any) for mail lost due to improper records or signatures.

3rd Party Deployment Profiles – Delegated

@DrKurtA

The 3rd party is also providing DNS, webhosting, etc for the Domain Owner and makes the process mostly transparent to the domain owner

Pro• Very easy for less sophisticated Domain Owners.• Can be mostly automated by the 3rd party.

Con• The domain owner is significantly more dependent on the 3rd party.• Can make interactions with more than one 3rd party impossible.

3rd Party Deployment Profiles – Hosted

@DrKurtA

Report Processing and Analysis

@DrKurtA

Report Parsing Tools

http://dmarc.org/resources.html

If you develop report parsing tools you are willing to share, please send a note to the dmarc-discuss list and let us know.

Report Processing and Analysis

http://dmarc.org/resources.html

@DrKurtA

Step 1: Categorize the IPs in the Aggregate Report

• Your Infrastructure

• Authorized 3rd Parties

• Unauthorized 3rd Parties *

* You should consider everything an Unauthorized 3rd Party by default; until proven otherwise

Report Processing and Analysis – Step 1

@DrKurtA

Step 2: Infrastructure Auditing

For both your Infrastructure and Authorized 3rd Parties

• Identify owner(s)

• Determine LOE for Deploying Domain Authentication

• Determine LOE for Identifier Alignment

• Evaluate or prepare business case / justification


@DrKurtA

Step 3: Identify Malicious Email

Research Unauthorized 3rd Parties and label the Abusers

• Use public data sources

• Vendor services

• Look out for known failure cases (SPF/DKIM/DMARC)

• Failure reports


@DrKurtA

Step 4: Perform Threat Assessment

Categories• Your Infrastructure• Authorized 3rd parties• Unauthorized 3rd parties• Abusers


@DrKurtA

Consider: • Phish vs. False Positives• Phish vs. Total Aligned Email

If there is no Phish, you don’t have a Domain Spoofing problem (at this time)


Phish Unaligned Email From Abusers

Definite False Positives

Unaligned Email from Your Infrastructure + Unaligned Email from Authorized 3rd parties

Potential False Positives

Unaligned Email from Unauthorized 3rd parties

@DrKurtA

Enforcement Policy Ramp-up

@DrKurtA

Step 1: Verify Authentication and Alignment for all of your

Infrastructure and all Authorized 3rd Parties

Step 2: Update your record to:p=quarantine; pct=10;

Do not:• Skip ‘quarantine’ and go straight to ‘reject’• Change the policy action from ‘none’ without setting a

‘pct’

Initial Policy Ramp-up

@DrKurtA


Step 3: Monitor your reports for issues and address them.

Make a ‘go forward / go back’ decision.

Step 4: Update your record to increase the ‘pct’.

Step n…: Rinse and repeat until you get to ‘pct=100’.

@DrKurtA


Step n+1: If needed and warranted by the abuse being seen, update your record to:

p=reject

@DrKurtA

Ongoing Monitoring

@DrKurtA

Ongoing Monitoring• Categorize new IPs in Aggregate reports

• Your Infrastructure• Authorized 3rd Parties• Unauthorized 3rd Parties• Abusers

• Reassess the Threat Level• Increases in phish• Changes in unaligned email volume• Make changes accordingly• Takedowns or other phish responses

@DrKurtA

Ongoing Monitoring

Be on the look out for:• Infrastructure changes

• New products / new subdomains

• New authorized 3rd parties

• Mergers and acquisitions

@DrKurtA

Protecting “me.example” – Q&A








@DrKurtA

Take Home Points In-house

- If email is important to your company and even more so if it is important to your product – email authentication is critical invest in doing it right and keeping it up to date

Out-sourced- Understand how your 3rd parties manage SPF (include values)

- Understand how they handle DKIM (pay extra if needed to get your own keys)

- Key rotation + length

- Understand how they work with DMARC and DMARC reports

@DrKurtA

Resources – Email Domain Authentication http://dmarc.org/resources

- Articles, Tutorials and Videos Online materials explaining DMARC and related or underlying email authentication technologies.

- Code and Libraries Modules and packages that you can deploy or build into programs that implement DMARC.

- Deployment Tools Tools and services you can use when deploying DMARC. Everything from DMARC DNS record builders, to message validators / reflectors that help you determine if DKIM/DMARC/SPF is working as expected.

- Products and Services Products and services that can assist you in deploying DMARC and related technologies. Everything from hosted email services, to email security firms, to mailing list management software.

http://dmarc.io

http://dmarc.org/resources

http://dmarc.org/resources/articles-tutorials-and-videos/

http://dmarc.org/resources/code-and-libraries/

http://dmarc.org/resources/code-and-libraries/

http://dmarc.org/resources/deployment-tools/

http://dmarc.org/resources/deployment-tools/

http://dmarc.org/resources/products-and-services/

http://dmarc.org/resources/products-and-services/

http://dmarc.io/

@DrKurtA

Resources – DNS(SEC), DANE, TLS, etc. http://www.internetsociety.org/deploy360/start/

https://www.ssllabs.com/ssltest/

http://docs.menandmice.com/display/MM/Where%20to%20find%20webbased%20DNSSEC%20testing%20tools

http://dnsviz.net/

http://valimail.com





http://dnsviz.net/

http://dnsviz.net/

http://valimail.com/

@DrKurtA

Points to Ponder

This is not “fire-and-forget”

In a dynamic environment, protecting your domain requires constant vigilance

What is your “bus number” for this critical activity?

More Questions?Office Hours

Friday - Lunch

@DrKurtA

[email protected]

What You Need to Know About Email Authentication

Technology

Transcript of What You Need to Know About Email Authentication