sSN08 SAN Best Practices
Transcript of sSN08 SAN Best Practices
-
8/13/2019 sSN08 SAN Best Practices
1/37
SAN Best Practices
Jim Blue
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
2/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
What will be covered
rc ec ure
Zoning
Fabric
Stora e
High availability
ange managemen
Solution-centric
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
3/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
rc ec ure
What is the purpose of the SAN?
How is the SAN supposed to help the business?
How critical is the data within the SAN?
a es gn an wea opera ona prac ces are
ingredients for failures
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
4/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
rc ec ure: y s mpor anStorage networking architecture may benefit the organization but you
must remember there are always trade-offs.
Goal Trade-off
High availability Needs redundant components / higher costs
Performance More equipment / higher costs
Security Monitoring requirements and reduced ease of use
Scalabilit Im act on availabilit and costs
High application throughput Other applications impacted
© 2011 IBM Corporation
,
-
8/13/2019 sSN08 SAN Best Practices
5/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Zonin : What is it and Wh use it?
Partition the SAN in logical groups of devices
Enables secure sharing of SAN resources
Optimize SAN resources and improves workload
balance
Limits the impact of failing devices
Adds flexibility to meet the needs of different end-users
Use LUN masking in conjunction with zoning; it is not a
re lacement for zonin
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
6/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Zonin : Methods
No zoning (worse case)
Zone by operating system
Zone by application
Zone by cluster groups
Zone b sin le initiator ort best scenario
© 2011 IBM Corporation
Never use WWNN for zone definitions!!!
-
8/13/2019 sSN08 SAN Best Practices
7/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Zonin : Namin
Consistent naming scheme for all components
Use meaningful names
Should define who, what, where and how Location
Server
Server port
Fabric
Example: PilRd_SQL12_ed74_DS8K1_1132_A
Location_server_HBA_storage_storage port_fabric
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
8/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Zonin : Se aration of traffic t es
Would you share tape and disk devices on same SCSI bus?
epara e ss m ar s s orage sys ems
Keep FCP and FICON separate
Reduces impact range of RSCNs
FICON does not need it
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
9/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Zonin : Multi athin
Works on the host system with multiple paths to thes orage sys em
Presents only one view of a logical volumeFail-over recovery
It IS possible to have too many paths
Too many paths will degrade overall performanceMore ports does not equal N x bandwidth increase
(Bandwidth increase depends on IO size but typically ranges from1.4 to 2.9 times increase when going from 1 to 4 HBAs)
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
10/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Zonin : Im lementin chan es
If available, use automated tools to verify proposed
changes
Make changes from the same switch
core-e ge a r c es gn, use a core sw c
Introduce changes during low I/O times
redundant fabric(s)
Approximately 3-5 seconds per active port
Remove zoning items not in use
Understand the default zone policy (deny or permit?)
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
11/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Fabric: Consistenc is the word
Naming schemeServers e.g. SQL12
Storage devices e.g. DS8k_1Port descriptors e.g. SQL12_ed74
Code/firmware levels on similar devicesCabling
Labels and fully documented
Use cable guides
o no m x ca e s zes or ype
(62.5 micron and 50 micron) (OM-2 and OM-3)
Limit use of patch panels per cable run
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
12/37
-
8/13/2019 sSN08 SAN Best Practices
13/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Fabric: Confi urations
Save device configurations before making any changesDo not store on SAN-attached storage
Fall-back mechanism
Version for the device
Another version which is human readable
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
14/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Fabric: Ex ansion
Use planning tools if available
Consult vendor interoperability matrices
If in doubt, open a proactive service ticket with vendor andask questions during the planning stage
Manual method: use number of active ports from the
Will the infrastructure need to grow?
Locally (expansion from growth and/or consolidation)
Distance (business continuance or disaster recovery)
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
15/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Stora e: Im lementation
Use stable code/firmware versions
Enable open WWPN access points on storage(if supported on the storage system)
Data replication?Don’t use the same storage port for host connections and
data replication
Do not share SVC and host traffic on the same storagedevice ort
Provide for remote access?
Your Administrators and vendor’s support personnel
v u u uServer definitions
Storage to server mappings (LUN masking)
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
16/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Stora e: Workload balance
Follow suggested fan-out ratio for storage device ports
If storage system has multiple ports, spread the
workload evenly
. .
Preferred paths
Failover scenarios
Boot from SAN scenario
Use available management tool to monitor workloads
, , ,
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
17/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Stora e: Workload balance
Must consider workload for performance
IO per second (for transaction workloads)
MB per second (for throughput workloads) Avera e size KB er IO
Read/write ratio
Sequential or random
High capacity disk does not fit all situations
Remember: Need to have similar info for ALL hosts, applications and
existing storage arrays
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
18/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Stora e: Workload balance
Beware of rank and/or RAID array loads
Use number of data drives to develop max load factor
Do not include parity and/or hot sparesDisk s eed determines workload factor
Rule of thumb (per disk drive). : – ps
10K RPM: 120 – 150 IOps
15K RPM: 165 – 200 IOps
SSD: 1500 IOps
(sequential read traffic will be less due to cache and pre-fetch)
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
19/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Stora e: How much is enou h?
Just-in-case provisioningExcess ca acit waitin to use
Typically higher operating expenses
Just-in-time rovisionin Add required capacity and allocate immediately
Requires very good planning
Either method may require data relocation
Understand the traffic pattern for a new project
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
20/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
g ava a y: unc ona es gn
o s ng e po nt o a ure
Very Small: Single or cascaded
Small: meshe um: par a mes
Large: core-edge
Keep high bandwidth consumers local
Storage and host(s) on same switch (core)
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
21/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
g ava a y: unc ona es gnBeware of ISL oversubscription (maximum 7:1 to 10:1)
(need to understand traffic patterns at higher ratios)
Use vendor’s suggested oversubscription ratio for inter-
(don’t exceed available bandwidth – fibre channel or LAN/WAN)
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
22/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Hi h availabilit : Servers and Clusterin
Redundant components in the servers
Beware of multiple applications with the same purposeCurrent device driver levels and O/S fixes
HA cluster is not the same as workload balancing
cluster or grid cluster
Clusters shift the focus from the server hardware to
the application
Cluster scale – local or dispersed?
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
23/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
g ava a y: or s r u on an res encyUsing trunking or port-channels?
prea e oa poss e
Servers connect to different switches or line modules
Spread storage system connections across
multiple switches and line modules
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
24/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Chan e mana ement: What is it and wh use it
Assess the risks of all proposed changes and decide if the
C/M does not have to be served by a dedicated application
C/M is a methodology that works for the business (end-
users) and IT team (implementers)When is a change not working and what actions are
needed to back-out / stabilize? (clear guidelines are a MUST!)
Use all available resources: Readme files, vendor, test
environment
© 2011 IBM Corporation
Verify all dependencies are known and in-place
-
8/13/2019 sSN08 SAN Best Practices
25/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Chan e mana ement: Documentation
Configurations of all devices (before and after)
Updated as part of the change process
Be able to explain decision for the changeDocumentation must be sufficient to build the SAN
environment from “clean slate” ( never stored on the SAN )
be simple and clear
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
26/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Chan e mana ement: Im lementation
Use out-of-band communications for management
ever use e au passwor s
If monitoring capability includes performance sampling,ca ture erformance baseline metrics re ularl
Establish threshold alerts for ISLs and high bandwidth edge
devices (varies from 66% to 75%) Automate processes where possible
Use dedicated management LAN or firewall protection
from all key systems for troubleshooting and recovery
(If in doubt, collect data anyway ASAP)
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
27/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
ange managemen : agrams
Can be created by a
variety of methods
n y a po n - n- me
image of the SAN
environment
Should show how and
where key components
are interconnected
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
28/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Chan e mana ement: Watch out for …
With many management applications, keep managed
environments to a minimum. Whether h sical or
virtual SANs
Provision for what is needed plus reasonable growth
Keep individual changes simple
Administrator roles assigned
Clear admin roles: Does HBA firmware/device driver
u dates belon to server or SAN administrator?
Before any change, verify all redundant paths are
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
29/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Solution-centric: Consistenc
Consult interoperability matrices and vendor support
Use NTP (network time protocol) where possible or set
device clocks to single time zone
uppor personne w an you
Quicker resolution of problems
Know when all ma or a lications are scheduled to run
Data backups
Data warehousing
Collect performance metrics on a regular basis
Naming convention
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
30/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Solution-centric: Mana ement a lications
Tools for complex environmentsRun on standalone s stems with internal stora e – do not use
SAN-attached resources
Uses standard interfaces/protocols
-
Assist the administrator with:
Change management
Configuration
Provisioning
Automation of routine tasks and procedures
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
31/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Solution-centric: Securit
Two types of security: data and access
Passwords
How strong is strong enough?
Save in a secure back-up repository
Are roles or levels of authority in use?
Data security includes encryption. Is there a
fallback mechanism to recover encrypted data or
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
32/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Solution-centric: General
If at all possible, create and use a testbed SAN
env ronmen
Time-sharing of development environment for
Changes to business-critical applications and
services chan es should be verified before roll-out
to the production environment
Develop a regular plan for code and firmware targetlevels for future upgrades
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
33/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Solution-centric: General
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
34/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
What was covered
rc ec ure
Zoning
Fabric
Stora e
High availability
ange managemen
Solution-centric
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
35/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
Special Session – s99GE
Storage Roundtable Congress Hall III Thursday, April 7
305 RAMAC
STORWIZE V7000
Bring your storage questions and ask the experts.Learn about IBM’s Storage roadmaps.
© 2011 IBM Corporation35
Non-disclosure form required!
-
8/13/2019 sSN08 SAN Best Practices
36/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
THANK YOU!
© 2011 IBM Corporation
-
8/13/2019 sSN08 SAN Best Practices
37/37
IBM Systems Technical University – Prague, Czech Republic, 4 – 7 April, 2011
IBM TotalStorage: SAN Product, Design, and Optimization Guide, SG24-6384
Brocade SAN Architectural Brief Brocade White Pa er GA-AB-023-00
Building and Scaling Brocade SAN Fabrics, Brocade Pub 53-0001573-01
Scalable Fabric Design – Oversubscription and Density Best PracticesCisco white paper ETMG 203154—LSK 05/04
, -
IBM TotalStorage DS8000 Series: Performance Monitoring and Tuning, SG24-7146
Veritas Dynamic MultiPathing (DMP) for Windows, Symantec White Paper, 07/2005
Brocade Multiprotocol Router SAN Design Guide, Brocade Pub 53-0000552-01
SAN Volume Controller: Best Practices and Performance Guidelines,SG24-7521
IBM TotalStorage Enterprise Tape: A Practical Guide, SG24-4632
Virtualization in a SAN, IBM Redbooks Paper REDP3633
VMware Virtual File S stem: Technical Overview and Best PracticesVMware white paper
HP StorageWorks: SAN Design Reference Guide, H-P Pub AA-RW86D-TE