Email Architecture - with sendmail and postfix · 1 Structure 2 Emailflows 3 Sendmail 4 Postfix 5...
Transcript of Email Architecture - with sendmail and postfix · 1 Structure 2 Emailflows 3 Sendmail 4 Postfix 5...
Email Architecture
with sendmail and postfix
Karst Koymans
Informatics Institute
University of Amsterdam
(version 1.7, 2010/10/01 11:41:11)
Tuesday, September 28, 2010
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 1 / 47
1 Structure
2 Email flows
3 Sendmail
4 Postfix
5 Other MTA’s
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 2 / 47
Structure
Outline
1 Structure
2 Email flows
3 Sendmail
4 Postfix
5 Other MTA’s
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 3 / 47
Structure
Organisation
Say ORG is an example organisation
AUT is an autonomous suborganisation
MAN is a managed suborganisation
aut is an autonomous part of MAN
man is a managed part of MAN
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 4 / 47
Structure
Structure of the organisation
ORG
AUT
MAN
aut
man
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 5 / 47
Structure
DNS mirrors the structure
Where are the cuts?
ORG.com.
AUT.ORG.com.
MAN.ORG.com.
aut.MAN.ORG.com.
man.MAN.ORG.com.
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 6 / 47
Structure
Email mirrors the structure
mail.* are mail relays and servers
mail.ORG.com.
mail.AUT.ORG.com.
mail.MAN.ORG.com.
mail.aut.MAN.ORG.com.
mail.man.MAN.ORG.com.
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 7 / 47
Structure
MX records (1)
ORG.com. MX 0 mail.ORG.com.
AUT.ORG.com. MX 0 mail.AUT.ORG.com.
10 mail.ORG.com.
MAN.ORG.com. MX 0 mail.MAN.ORG.com.
10 mail.ORG.com.
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 8 / 47
Structure
MX records (2)
aut.MAN.ORG.com. MX 0 mail.aut.MAN.ORG.com.
5 mail.MAN.ORG.com.
10 mail.ORG.com.
man.MAN.ORG.com. MX 0 mail.man.MAN.ORG.com.
5 mail.MAN.ORG.com.
10 mail.ORG.com.
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 9 / 47
Structure
Email addresses
Employee “The Boss” working in department “aut”
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 10 / 47
Structure
Email forwarding
[email protected] is forwarded to
[email protected], which is in turn forwarded to
Forwarding can be
user based (.forward)
system based (alias file or database)
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 11 / 47
Email flows
Outline
1 Structure
2 Email flows
3 Sendmail
4 Postfix
5 Other MTA’s
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 12 / 47
Email flows
SMTP flow (inbound) (1)
Directly to mailhost in MX record
[email protected] enters at top
[email protected] enters at leaf
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 13 / 47
Email flows
SMTP flow (inbound) (2)
Always to mail.ORG.com.
Requires “split” DNS
Different outside MX record for aut.MAN.ORG.com., pointing to
mail.ORG.com.
Alternatively block port 25 from the outside
What is your opinion about this solution?
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 14 / 47
Email flows
SMTP flow (outbound)
Directly to outside world
No “corporate” policy
Needs smart hosts decentrally
Flowing up the tree step by step
Uses the “smart host” option
Directly to the top of the tree
Uses the “smart host” option
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 15 / 47
Email flows
Mail access
Only leaf mail servers supply mail access
Intermediate servers are relay only
In case you want to deliver higher in the tree
Create an extra child for mail delivery
Separate SMTP relay from local delivery and IMAP access
Possibly separate outgoing and incoming email,
much like separating authoritative and caching name servers
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 16 / 47
Sendmail
Outline
1 Structure
2 Email flows
3 Sendmail
4 Postfix
5 Other MTA’s
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 17 / 47
Sendmail
sendmail configuration (for Ubuntu 10.04.1 LTS)
Debian specific (based on sendmail 8.14.3)
Has an extensive init script to control sendmail execution
Uses a separate sendmail.conf file to source inside init script
Uses a helper program (sendmailconfig) to generate
the main configuration file sendmail.mc
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 18 / 47
Sendmail
sendmail configuration directory
/etc/mail as configuration directory
sendmail.mc, which is used to generate
sendmail.cf
using the m4 macro processor
local-host-names
aliases
access
. . .
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 19 / 47
Sendmail
m4 macros
Inside /usr/share/sendmail/cf
m4 source files m4/*
cf.m4, cfhead.m4, proto.m4
debian/*, domain/*, feature/*, . . .
hack/*, mailer/*, ostype/*, . . .
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 20 / 47
Sendmail
sendmail.mc
OSTYPE(debian)
DOMAIN(debian-mta)
DAEMON_OPTIONS(. . . )
FEATURE(. . . )
no_default_msa
access_db
. . .
MAILER
local
smtp
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 21 / 47
Sendmail
debian.m4
define(conf. . . )
Lots of configuration parameters, to name a few
confSMTP_LOGIN_MSG
confCW_FILE
confDEF_USER_ID
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 22 / 47
Sendmail
debian-mta.m4
Many more conf. . . options
confMAX_HOP
confDONT_BLAME_SENDMAIL
All kinds of TimeOut(TO)-timers
confTO_MAIL
confTO_QUIT
. . .
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 23 / 47
Sendmail
sendmail.cf macros
Macros
C<class>
referenced as $=<class>
F<class_in_file>
for example Fw/etc/mail/local-host-names
also referenced as $=<class_in_file>
D<name>
referenced as $<name>
Classes and names are often single letter identifiers
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 24 / 47
Sendmail
sendmail.cf hostnames
sendmail -bt -d0.4
for debugging local hostname(s)
Dj$w.$m (set internally; automatically)
hence $j=$w.$m
What is inside $=w class?
Many “hostnames”, also numeric
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 25 / 47
Sendmail
sendmail.cf map lookup
K<mapname> <type> <detail>
mailertable hash -o /etc/mail/mailertable.db
generics hash -o /etc/mail/genericstable.db
virtuser hash -o /etc/mail/virtusertable.db
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 26 / 47
Sendmail
sendmail.cf options
AliasFile
ForwardPath
DaemonPortOptions (UseMSP)
Timeout
*LA (Queue, Refuse, Delay)
SmtpGreetingMessage
. . .
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 27 / 47
Sendmail
sendmail.cf headers
HReceived:
$?sfrom $s $.$?_($?s$|from $.$_)$.
$?{auth_type}(authenticated$?{auth_ssf} bits=${auth_ssf}$.)$.
by $j ($v/$Z)$?r with $r$. id $i
$?{tls_version}(version=${tls_version} cipher=${cipher}
bits=${cipher_bits} verify=${verify})$.
$?u for $u; $|; $.$b
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 28 / 47
Sendmail
sendmail.cf mailers
M<mailer> <attributes>
local (maybe procmail as MDA)
prog, *file*, *include* (builtin)
smtp, esmtp, smtp8, relay, bsmtp, fido
procmail (as mail filter, called with “-m”)
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 29 / 47
Sendmail
sendmail.cf rulesets and rules
S<name>=<number> indicates a ruleset
canonify=3 (always first)
parse=0 (resolves <mailer,host,user>)
check_relay (to disable open relaying)
check_mail (checks MAIL FROM:)
check_rcpt (checks RCPT TO:)
R<LHS> −−→−−→ <RHS> −−→−−→ <comment> indicates a rule
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 30 / 47
Sendmail
sendmail.cf rules
LHS (Left Hand Side)
$*, $+, $- (token matching)
$@ (matching zero tokens, used for empty input)
$=, $˜(class matching, positive or negative)
RHS (Right Hand Side)
$1, $2, . . . (substitution of matched parts)
$:, $@ (control flow)
$>, $?$|$. (recursion; conditional)
$[. . . $], $(. . . $) (IP lookup; map lookup)
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 31 / 47
Sendmail
Sendmail ruleset testing
Command Use for
=S<ruleset>, =M showing rulesets and mailers
$<m>, $=<c> evaluating macros
/parse <address> address parsing
/try <mailer> <address> address given to mailer
/map <map> <lookup> map lookup
Table: Some sendmail -bt commands
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 32 / 47
Sendmail
A new kid on the block?
Projects to modernise sendmail
Sendmail X, which has been succeeded by
MeTA1
Resembles Postfix and qmail in architecture
All processes are resident and not semi-resident
(as in Postfix) or on demand (as in qmail)
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 33 / 47
Postfix
Outline
1 Structure
2 Email flows
3 Sendmail
4 Postfix
5 Other MTA’s
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 34 / 47
Postfix
Postfix
(Mostly) compatible with sendmail
supplies /usr/{lib,sbin}/sendmail emulation
Good performance
Safe and secure
Modular and flexible
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 35 / 47
Postfix
General postfix features
Support for multiple transports
Easy virtual domain configuration
Extensive UCE/SPAM control
Rewriting through table lookups
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 36 / 47
Postfix
Postfix modular setup
One resident master process
compare to inetd super server
Some semi-resident daemons
started via master.cf file
something like inetd.conf
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 37 / 47
Postfix
Postfix queues
maildrop (local incoming)
incoming (after cleanup)
active (being worked on)
deferred (temporary failure)
hold (needs human intervention)
corrupt (needs human inspection)
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 38 / 47
Postfix
Postfix security
Runs not setuid root
Uses chroot environment
Is modular and not monolithic
Filtering of outside information
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 39 / 47
Postfix
Postfix daemons
pickup (mail from maildrop via postdrop (“sendmail”))
smtpd (remote mail from the Internet)
cleanup (repairs incoming mail)
qmgr (processes mail queues)
local (local delivery)
smtp (remote delivery)
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 40 / 47
Postfix
Postfix assistants
(trivial-)rewrite
canonicalisation (compare “ruleset 3”)
resolving (compare “ruleset 0”)
bounce
error mailer
defer messages
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 41 / 47
Postfix
Postfix/Sendmail tables
Postfix Sendmail
virtual virtusertable
canonical genericstable
transport mailertable
access access
relocated - (aliases)
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 42 / 47
Postfix
Postfix architecture inbound
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 43 / 47
Postfix
Postfix architecture outbound
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 44 / 47
Other MTA’s
Outline
1 Structure
2 Email flows
3 Sendmail
4 Postfix
5 Other MTA’s
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 45 / 47
Other MTA’s
qmail
Who looked at qmail and wants to explain?
Karst Koymans (UvA) Email Architecture Tuesday, September 28, 2010 46 / 47