Exchange Storage Sizing and Hardware Exposed

Jim McBee

http://www.ithicos.com

Who is Jim McBee!!??• Consultant, Writer, MCSE, MVP and MCT –

Honolulu, Hawaii• Principal clients (Dell, Microsoft, SAIC, Servco

Pacific)• Author – Exchange 2003 Advanced

Administration (Sybex)• Contributor – Exchange and Outlook

Administrator• Blog

● http://mostlyexchange.blogspot.com ● http://www.directory-update.com

Agenda

• The sizing quandary • User usage profiles• Microsoft’s hardware recommendations• Network sizing• RAM sizing• CPU sizing• Disk sizing• Did you get it right?

The sizing quandary

Typical solution

• Throw hardware at the solution…

• Sometimes you get it right… ● and sometimes you don’t…

• Often disk capacity is still undersized

Usage profiles

Messaging user typesType Messages

sent/receivedMailbox size

Light 5 sent

20 received

50MB

Average 10 sent

40 received

100MB

Heavy 20 sent

80 received

200MB

Large 30 sent

120 received

500MB

Active Directory

Sizing domain controllers• Use Windows 2003 SP1 if possible• Windows 2003 domain and forest functional level• Use 4:1 ratio

● For each 4 Exchange CPUs/cores use 1 DC/GC core● This ratio assumes dedicated Exchange DCs

• Redundant DNS servers and domain controllers• Larger Active Directories…

● Put transaction logs and databases on different spindles● If NTDS.DIT database exceeds about 1GB, use Windows the

/3GB /USERVA=3030 switches● If the NTDS.DIT database exceeds about 2.5GB, use Windows

2003 x64 or Windows 2008 x64

Problem indicators

• MSExchangeDSAccess Process – E2K3

• MSExchange ADAccess Domain Controllers● LDAP Read Time● LDAP Search Time

• Should be below 50 milliseconds on average• Should never exceed 100 milliseconds

Network sizing

Network connections

• 100Mbps / Full Duplex

• Switched backbone is best

• Ensure that network adapter is inserted in to network at maximum speed possible● May require manual configuration on server

and switch

• Use separate 1GB Ethernet connections for iSCSI

Key indicator you may have network problems

• Performance monitor:● MSExchangeIS / RPC Averaged Latency

• Should be below 50 milliseconds• May indicate hardware problems, but network is

often the culprit

● Network Interface / Bytes Total/sec• Watch for this counter to approach maximum

network bandwidth

CPU sizing

Microsoft’s recommendation

Server role Minimum Recommended Maximum

Mailbox 1 CPU core 4 CPU cores 8 CPU cores

Client Access 1 CPU core 4 CPU cores 4 CPU cores

Hub Transport 1 CPU core 4 CPU cores 8 CPU cores

Unified Messaging

1 CPU core 4 CPU cores 4 CPU cores

Combined function

1 CPU core 4 CPU cores 4 CPU cores

Edge Transport 1 CPU core 2 CPU cores 4 CPU cores

Megacycles per second

• Number of MHz that are required for a particular server

• Take an average CPU usage during the busiest part of the day

• Megacycles per mailbox = (Avg CPU * Speed of processors in MHz * Number of processors) / Number of mailboxes

Excessive CPU usage?

• Average processor usage stays above 70 – 80% during the busiest parts of the day

Memory sizing

RAM is the key to improving performance in Exchange

• Exchange 2003● 4GB of RAM per server● User /3GB /USERVA=3030 in BOOT.INI file● Large environments● 2GB – 4GB of RAM for front-end servers and

bridgehead servers

• For tuning information…● See KB 815372 How to Optimize Memory

Usage in Exchange Server 2003

Exchange 2003 and caching

• Exchange 2003 Extensible Storage Engine maximum cache● 1.2GB of cache● 576MB or 896MB by default

• 500 users on an optimized E2K3 server● 2.4MB of cache per user

• 1000 users on an optimized E2K3 server● 1.2MB of cache per user

• 2000 users on an optimized E2K3 server● .6MB of cache per user

Exchange 2007 memory sizingServer role Minimum Recommended Maximum

Hub Transport 2GB 1GB per CPU core (2GB minimum)

16GB

Client Access 2GB 1GB per CPU core (2GB minimum)

8GB

Unified Messaging

2GB 1GB per CPU core (2GB minimum)

4GB

Edge Transport 2GB 1GB per CPU core (2GB minimum)

16GB

Mailbox server roles

• Minimum of 2GB of RAM● Minimum of 3GB of RAM if LCR is used● Recommend 4GB of RAM

• Recommend 2GB + 5MB per mailbox for heavy users

• Can size memory based on number of storage groups, too. ● Add 2GB for each four storage groups● Take whichever value is higher

• BOOT.INI optimization is not necessary• 32GB maximum practical amount

Cache is essential

• If you follow recommendations for heavy users, each user will ALWAYS have 5MB of cache available

• Sufficient memory is essential to get expected disk performance

• Less available cache = More frequent disk I/O

Disk sizing

Introducing IOPS

• IOPS = Disk I/O operations per second● Pronounced eye-ops

• Typical I/O capacity● 10,000 RPM SATA drive = 70 IOPS● 10,000 RPM fiber channel disk = 100 IOPS● 15,000 RPM fiber channel disk = 120 IOPS● 15,000 RPM SAS drive = 170 IOPS

Exchange 2003 IOPS Estimates

User type Database volume IOPS

Light .5

Average .75

Heavy 1.0

Large 1.5

Exchange 2003 diminished cache performance

Mailboxes Estimated IOPS

Calculated IOPS Required

Actual IOPS Required

1000 1.0 1000 1000

2000 1.0 2000 2500

3000 1.0 3000 3750

5000 1.0 4000 5000

Why? Less cache per mailbox

Exchange 2007 IOPS Estimates

User type Database volume IOPS

Light .14

Average .20

Heavy .27

Large .41

*Assumes you are following Microsoft’s memory recommendations for Exchange 2007

Factors that affect IOPS

• Antivirus software• Backups• Outlook in cached mode• LCR (estimates are 150%

of database IOPS)• Full text indexing (10%

overhead for E2K7)• Forms processing, work

flow systems, or unusual messaging spikes

• Online maintenance• Other server roles on

same hardware (such as Hub Transport)

• Paging• Connectivity / protocol

logging• Messaging records

management

Transaction Logs and IOPS

• Microsoft’s estimates are for databases only• Transaction logs may generate 10 – 25%

additional I/O

Viewing Disk Transfers/sec

You can help!

• Use the maximum amount of practical memory for Exchange mailbox servers

• Sizing disk on right sector● Use DiskPart.exe● See KB 300415

• Configure caching controllers for 75% write / 25% read

Signs of a disk bottleneck

• Physical disk’s Avg. Disk sec/Transfer > .02 on a sustained basis

• Physical disk’s Disk Transfers/sec staying at a sustained level

Sustained I/O activity

Tools you can use

• Performance monitor• Exchange Best Practices Analyzer• Performance Tuning Analyzer• Exchange Server JetStress• Exchange Server Load Generator• Exchange Server Stress and Performance• Exchange Troubleshooting Assistant• User Monitor• Microsoft Operations Manager

Free eBook

• The Shortcut Guide to Exchange Server 2007 Storage Systems

● http://nexus.realtimepublishers.com/SGES2K7SS.htm

Questions?

Thanks for attending!

Book giveaway and e-mail notice

• Please give me a piece of paper with your name for drawing

• Include your e-mail address or give me a business card if you want:

● 20% discount code for Directory Update software

● Notification e-mail when Mastering Exchange Server 2007 is available

Your Feedback is Important

Please fill out a session evaluation form and either put them in the basket near the exit or drop them off at the conference registration

desk.

You could win one of 10 subscriptions to TechNet Plus Direct: The essential resource for IT Professionals .

Winners will be drawn and names will be posted Tuesday morning from Monday evals, Wednesday morning from Tuesday evals, and during closing session from Wendesday evals. Include your badge number

on your session eval so we can figure out the winners!

Thank you!