© 2009 Quest Software, Inc. ALL RIGHTS RESERVED

For audio, please call the phone number in your invitation email. Audio will not come through your speakers.

SQL Server onStorage Area Networks

2

Brent Ozar• SQL Server MVP

for Quest Software• Managed SQL, VM,

80tb SAN for $7b co• Published author• Made lots of

expensive mistakes• www.BrentOzar.com• @BrentO on Twitter

3

BrentOzar.com/go/insidetheblackbox

4

The Agenda• Inside the SAN• DBA Pros & Cons• Starting From The

Beginning• How to Use SQLIO• Multipathing

5

In The Beginning…

6

Next: External Storage

7

Storage Area Networks• Independent• Expandable• Well-Protected• Blazing Fast• Complex• Expensive

8

9

Photo Licensed with Creative Commons from http://www.flickr.com/photos/tophost/2247031208/

10

Photo Licensed with Creative Commons from http://www.flickr.com/photos/tophost/2246209421/

11

Photo Source http://www.flickr.com/photos/tophost/2246209421/

12

SAN Components• Drive enclosures• Controllers (& Cache)• Switch networks• Host Bus Adapters• HBA Drivers• Servers

13

Drive Allocation MethodsShared Dedicated

Grow Volumes In…

Tiny Increments

Whole Drives

Drives Idle… Rarely Often

Performance Management

Easy Hard

Overall Speed Fast Enough Really Fast - Maybe

14

Enterprise Midrange Entry-Level

Uptime 100% >99% >99%

Features Stable Growing Exploding

Interface Fiber & iSCSI

Fiber or iSCSI

iSCSI

Entry Cost $1,000,000 $200,000 $20,000

Vendors EMC HitachiHP EVAIBM

CompellentEMCIBM NetApp

Dell (Equallogic)LeftHand

15

Good Things• Clustering• Table partitioning• Faster builds• Easier growth• Caching speed

Photo Source http://www.flickr.com/photos/pcka/3240903696/

16

Not-So-Good Things• Black box• Caching

may not help• Compatibility lists• More drivers• Tough to grow• SAN admins

may not help

Photo Source http://www.flickr.com/photos/stewf/95709873/

17

“How many gigs do you need?”

18

“We’ll keep an eye on it for ya.”• Perfmon changes• Neighbors change• SAN changes• SAN monitoring

software is tough

Photo Source http://www.flickr.com/photos/mrjoro/39470992/

19

What’s Your Bottleneck?• Not Enough Drives• Drives Too Slow• Shared Drives• RAID Level• SAN Controllers• Partition Alignment• Drivers• Multipathing

20

Things to Monitor• Hard drives• Controllers (CPU, caching)• Controller connectivity to

the network• Network switch throughput• Server connectivity to the

SAN (the HBA) • Server drivers

21

The X Factor: Redundancy• Hard drives – hundreds• Controllers – 2+• Controller connectivity to network – 4+• Network – separate networks, switches• Server connectivity to SAN – 2+• Network fabrics – iSCSI, Fiber, WAN

22

SAN Monitoring• Strict HCLs, SCLs• Cross-platform

means JAVA• No “full” picture• Very, very, very

expensive

Photo Source: http://flickr.com/photos/earthandeden/865393933/

23

Unpredictable, Unexplainable

24

Start From The Beginning• Get block sizes right• Align partitions• Design for pathing• Test with SQLIO• Install SQL Server• Monitor sec/read, write• Save history

25

Simple SAN Pathing

26

Simple SAN Pathing

27

Simple SAN Pathing

28

Simple SAN Pathing

29

SIMPLE?!?1#%&@!?

30

Photo Licensed with Creative Commons from http://www.flickr.com/photos/creatista/204265480/

31

Goals of Multipathing

1. Protection

2. Performance

32

Active/Active Can Mean

33

Active/Active Gotchas• One path per LUN/volume• Sending vs receiving• Post-failover stickiness• Network design

34

Network Bottleneck Example

35

Pathing Really Matters Now

36

Photo Licensed with Creative Commons from http://www.flickr.com/photos/thatguyfromcchs08/2300190277/

37

Photo Licensed with Creative Commons from http://www.flickr.com/photos/atomdocs/3127663764/

Why Can’t We Go Faster?

38

How to Run SQLIO

sqlio -kW -t2 -s120 -dM -o1 -frandom -b64 -BH -LS Testfile.dat

39

Write This Down. It’s Important.

sqlio -kW -t2 -s120 -dM -o1 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t2 -s120 -dM -o2 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t2 -s120 -dM -o4 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t2 -s120 -dM -o8 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t2 -s120 -dM -o16 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t2 -s120 -dM -o32 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t2 -s120 -dM -o64 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t2 -s120 -dM -o128 -frandom -b64 -BH -LS Testfile.dat

sqlio -kW -t4 -s120 -dM -o1 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t4 -s120 -dM -o2 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t4 -s120 -dM -o4 -frandom -b64 -BH -LS Testfile.datsqlio -kW -t4 -s120 -dM -o8 -frandom -b64 -BH -LS Testfile.dat

40

We Did the Hard Work

41

The OutputE:\Program Files (x86)\SQLIO>sqlio -kW -t2 -s120 -dM -o1 -frandom -b64 -BH -LS Testfile.dat sqlio v1.5.SGusing system counter for latency timings, -1361967296 counts per second2 threads writing for 120 secs to file M:Testfile.dat

using 64KB random IOsenabling multiple I/Os per thread with 1 outstandingbuffering set to use hardware disk cache (but not file cache)

using current size: 24576 MB for file: M:Testfile.datinitialization doneCUMULATIVE DATA:throughput metrics:IOs/sec: 1539.50MBs/sec: 96.21latency metrics:Min_Latency(ms): 0Avg_Latency(ms): 0Max_Latency(ms): 572histogram:ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+%: 66 32 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

42

Common Bottlenecks• 1 gig = ~125 MB/sec• 2 gig = ~250 MB/sec• 4 gig = ~500 MB/sec• 10 gig = ~1250 MB/sec

43

Working Around It• Data/logs/tempdb

on different volumes• Separate

groups of DBs• Separate DB

into filegroups• TempDB, backups

Photo Source http://www.flickr.com/photos/doodlemonger/2667917978/

44

Wrapping It Up• Test before

deployment• Learn multipathing• Design DB files

around SAN limits• Monitor forever• Get vendor help

Photo Source http://www.flickr.com/photos/doodlemonger/2667917978/

45

Advanced Topics• Table partitioning• Filegroups & files• Snapshot backups• Storage virtualization• Storage VMotion• SAN administration

Photo Source http://www.flickr.com/photos/thomasthomas/504369245/

46

BrentOzar.com/go/insidetheblackbox