Storage Virtualization with IBM SAN Volume Controller · Storage Virtualization with IBM SAN Volume...
Transcript of Storage Virtualization with IBM SAN Volume Controller · Storage Virtualization with IBM SAN Volume...
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 1
Storage Systems Division
DECUS IT-Symposium 2007 | 1G04 © 2007 IBM Corporation
Storage Virtualization withIBM SAN Volume Controller
Torsten RothenwaldtIBM European Storage Center
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation2
Agenda
� Review different architectures of disk block virtualization.
� Explain implementation details of the IBM SAN Volume Controller as example of in-band appliance.
� Discuss current enterprise usage of disk/file virtualization.
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 2
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation3
Table of contents
Disk block virtualization
IBM SAN Volume Controller
Storage virtualization in enterprises today
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation4
Disk (Drive) Virtualization
� Physical data layout
Cylinder/Head/Sector addresses
� Media defects
� Virtual data layout
Logical Block Addresses (LBA)
� “Defect-free”
0000
0001
0002
0003
….
nnnn
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 3
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation5
Block-Level Virtualization
� Bound to a physical machine
� Fixed size and (pseudo-) geometry
� Limited performance
� Do break (occasionally)
� Anywhere (can be moved or replaced)
� Size as needed (can grow, shrink, or morph)
� Performance scaling
� Reliable as needed
Where does block-level virtualization reside?
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation6
Virtualization in the Storage Subsystem
+Heterogeneous hosts+Mature industry & products
�Performance �Stable & reliable�Perceived security
-Homogeneous storage-Management
�Device-specific�“per-box”
Virtualization
RAID subsystem
Per-host
Provisioning
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 4
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation7
Virtualization in the Host
+Flexibility (any storage)+File system coupling(online growth, re-layout, movement, snapshots,…)
-Server-centric management �of RAID subsystems�of volumes
-Complexity �shared data clusters
Virtualization
Per-host…still
Provisioning
Host
RAID subsystem
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation8
“Out-of-Band” Virtualization
+Data center-wide management+Shorter I/O path+Light-weight compared to full volume manager
-May be host-invasive �Host-specific
-Appliance availability-Secure protocols
Virtualization
Data center-wide
Provisioning
Host
RAID subsystem
“SAN appliance”�separate box�In-host
Infrastructure
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 5
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation9
Virtualization in the Network: “In-Band”
+Data center-wide management �Heterogeneous storage�Heterogeneous hosts
-Complexity �User (un)familiarity �Integration needs �Needs clustering
-Performance perceptions
Virtualization
Data center-wide
Provisioning
Host
RAID subsystem
SAN�Host access�Device access
Infrastructure
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation10
Virtualization Devices for In-Band
� Server-based device (appliance)
Virtualizes a variety of physical storage using different HBAs. One pool.
Implements complex storage solutions inexpensively
Adds another layer (managed separately)
Less interoperability issues (no fabric integration)
Appears as standard device.
� Switch-based device (fabric application)
Network optimized
High port counts
Expensive
Elaborated functions not available
Fabric integration critical
Standardization just beginning (ANSI T11 Fabric Application Interface Standard)
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 6
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation11
Table of contents
Disk block virtualization
IBM SAN Volume Controller
Storage virtualization in enterprises today
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation12
Example of a Block-Level In-Band Appliance:IBM SAN Volume Controller
� J. S. Glider, C. F. Fuente, W. J. Scales:The software architecture of a SAN storage control system.IBM Systems Journal, Vol. 42 (2003) Nr. 2, pp. 232-249
( http://www.research.ibm.com/journal/ )
� IBM Redbooks:
SG24-6423 IBM System Storage SAN Volume Controller
SG24-7275 Implementing the SVC in an OEM Environment
SG24-7371 Using the SVC for Business Continuity
( http://www.redbooks.ibm.com/ )
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 7
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation13
IBM SVC: Architecture
StorageStorageNetworkNetwork
NodeNode
Managed Disks
Virtual LUNs Virtual LUNs Virtual LUNs Virtual LUNs
NodeN ode NodeNode NodeNode NodeN ode N odeNodeNodeN ode NodeNode
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation14
IBM SVC: Zoning
Host Zone:-Hosts zoned only to SVC-See only Virtual Disks that SVC allows to see
Device Zone:-Devices zoned only to SV-See only SVC nodes as connected hosts
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 8
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation15
IBM SVC: Managed Disks
R5
VLUNR10 VLUN
R5
VLUN
R1
VLUN
R5
VLUN
R1
VLUN
RAID Controller RAID Controller
SANVC
R5
MD
R10
MD
R5
MD
R1
MD
R5
MD
R1
MD
� SVC does not perform RAID functions� Utilizes RAID capability of backend storage server� RAID-5, RAID-10, or RAID-1 recommended� Normally LUNs "surfaced" from storage systems are what the hosts on the SAN see as physical disks
� Disks surfaced by the storage systems are discovered by SVC and referred to as Managed Disks (mdisks)
� Spare capacity on mdisks can be reallocated transparently and dynamically
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation16
IBM SVC: Managed Disk Groups
R5R5
R5 Pool
...
Extent 1
Extent 2
Extent 3
Extent 4
Extent 5
Extent n
R1
R1 R1
R1 Pool
R10 R10R10
R10 Pool
R5
� Once the mdisks are available to SVC, the user assigns them to one or more pools called Managed Disk Groups
� These MDGs are addressed by the SVC in terms of extents:Extent size is determined at MDG creation time, default 16 MB, max 512 MB
Extent size determines maximum amount of storage SVC can manage
16-MB extents = 64 TB, 512-MB extents = 2 PB
Cannot migrate vdisks between MDGs with different extent sizes
Can migrate extents from mdisk to mdisk
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 9
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation17
IBM SVC: Virtual Disks
Extent 1a
Extent 1b
Extent 1c
Extent 1d
Extent 1e
Extent 1f
Extent 1g
Extent 2a
Extent 2b
Extent 2c
Extent 2d
Extent 2e
Extent 2f
Extent 2g
Extent 3a
Extent 3b
Extent 3c
Extent 3d
Extent 3e
Extent 3f
Extent 3g
Extent 1a
Extent 2a
Extent 3a
Extent 1b
Extent 2b
Extent 3b
Extent 1c
Extent 2c
Extent 3c
Create a striped
virtual disk
A host vdisk is a collection of Extents —each 16 MB - 512 MB
� From these extents, the user can build "virtual disks"� Various policies can be used to build them:
Striped — taking an extent in turn from each disk in the pool or a subset of the disks in the pool (Virtual Disk striped across multiple disks)
Sequential — using a single disk in the pool (Virtual Disk mapped sequentially to managed disk)
Image Mode — Virtual Disk = Physical LUN
SVC Cache — Enabled or Disabled
� Real physical capacity must be available to create a vdisk� Virtual disks can be expanded, reduced, or deleted� IO governing can be enabled to limit IO/s or MB/s
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation18
Cache Disabled Virtual Disks
� Can be used with Image Mode VDisks only to enable use of copy services of underlying disk subsystem
� Can be used with striped mode VDisksMay be useful to preserve cache resources
Example: Use for TSM/HSM disk pools
Example: Use for test/dev VDisks in mixed production environment
� Cache Disabled Virtual Disks are basically normal VDisks and can utilize SVC capabilities like migration and copy servicesCan use SVC copy services on the source VDisk
Cannot use SVC copy services on the target VDisk
� Provide a way for parallel direct access to the underlying disk subsystem
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 10
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation19
IBM SVC: IO Groups and Hosts
HostVDisk
WWPN1
WWPN2
WWPN3
WWPN4
IO Group
� Each virtual disk is assigned to a particular IO Group (node pair).
Every node in the cluster is aware of the virtual disk, but only owning IO Group services requests.
All IO is targeted at either of the nodes in the IO Group for purposes of caching and load balancing.
� It is these virtual disks that SVC presents to hosts on the SAN as targets of IO.The virtual disks are mapped to hosts (SDD for mutli-path operation).
Can be mapped to multiple hosts for use with clustering software.
The hosts see these as physical disks (in terms of the OS).
SVC knows hosts as groups of HBA WWPNs.
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation20
SVCCluster
I/O Group 1
VirtualDisk
LUN
ManagedDisk
mdisk0
10GB
mdisk1
10GB
mdisk2
10GB
mdisk3
10GB
mdisk6
20GB
mdisk5
20GB
mdisk4
20GB
FAStT
10GB
FAStT
10GB
FAStT
10GB
FAStT
10GB
ESS
20GB
ESS
20GB
ESS
20GB
ManagedDisk Groups
mdiskgrp0 [FAStT Group] - 40GB mdiskgrp1 [ESS Group] - 60GB
vdisk0
20GB
vdisk1
20GB
vdisk2
20GB
vdisk3
20GB
vdisk4
20GB
Virtual Disks Mapped to Hosts
SVC Combined Physical & Logical View
I/O Group 2
Virtual Disks are associated with particular I/O Groups
Managed Disk Groups are accessible by all I/O Groups in the Cluster.
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 11
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation21
IBM SVC: Clustering
�Cluster comprised of 2…8 nodes but administered as single image
�No Linux clustering software, failover/failback is function of SVC code
�One node automatically designated config/boss node for cluster
� Assigned cluster IP address
� Responsible for coordination of node transitions
� Automatic failover of config node and IP address
�Auto-restart of a node on failure and re-admission to cluster
�Cluster requires majority of nodes remain operating to ensure quorum
�Two nodes is a special case
� Quorum disk is elected as a tie-breaker
� Quorum disks are existing mdisks under SVC and use 1 extent
�A node stores a write in its own write cache and the write cache of its partner node before acknowledging to the host application
�On node failure, surviving node empties write cache and proceeds in write-through mode
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation22
RAIDcontroller 2
RAID controller 1
LUN 4
LUN 3
LUN 2
LUN 1
LUN 4
LUN 3
LUN 2
LUN 1
MD 4
MD 3
MD 2
MD 1
MD 8
MD 7
MD 6
MD 5
VD 4
VD 3
VD 2
VD 1
VD 7
VD 6
VD 5
IO Group
IBM SVC: Copy Services
RAIDcontroller 4
RAID controller 3
LUN 4
LUN 3
LUN 2
LUN 1
LUN 4
LUN 3
LUN 2
LUN 1
MD 4
MD 3
MD 2
MD 1
MD 8
MD 7
MD 6
MD 5
VD 4
VD 3
VD 2
VD 1
VD 7
VD 6
VD 5
IO Group
SAN Peer-to-Peer Remote Copy (PPRC) “outside the box”(Metro / Global Mirror)
SAN Data Migration “outside the box”SAN FlashCopy “outside the box”
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 12
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation23
IBM SVC Global Mirror
� Long distance asynchronous remote mirroring function
� Practically unlimited distances for business continuity
� Does not wait for secondary I/O before completing host I/O
Minimizes performance impact to applications
� Designed to maintain consistent secondary copy at all times
Once initial copy has completed
� Built on Metro Mirror code base
� Metro and Global Mirror delivered as single feature
Offers great implementation flexibility
No additional license chargefor existing MM users
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation24
Global Mirror: Overview
� Single remote copy partnership per cluster� Supports consistency groups that span I/O groups within cluster� Consistency group consists of MM or GM relationships not both
Consist-ency
Group
Cluster 1 Cluster 2Partnership
Vdisk
A
Vdisk
B
Vdisk
C
Vdisk
D
Relationship
Relationship
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 13
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation25
Global Mirror: Algorithm� I/O complete acknowledged to host at primary once following occurs:
Data is committed to cache in both nodes in I/O group
Log of I/O and its sequence number committed on both nodes of I/O group
� Remote I/O is transmitted concurrently with processing of local I/OLocal host I/O normally completes before remote I/O completes
� Extra messages passed between nodes in clusterUsed to detect concurrent host I/O among VDisks in same consistency group
� Concurrent I/O is assigned a shared “sequence number”
During high loads many I/Os will share sequence number to minimize overhead
� Remote I/Os are applied at secondary in “sequence number” order
Writes within a “sequence number” are non-dependant and written in any order once received at secondary
Many sequence number “batches” can be outstanding on the long distance link
At secondary site a single sequence number is applied at a time
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation26
Global Mirror: Link Tolerance
� Existing redundancy used to continue operation if node or link fails:
Recovery begins by re-driving writes that were in flight.
Log of I/O established during initial host write used to support recovery.
Recovery uses read to primary VDisk to capture data for replay of write
� Autonomic Link Tolerance feature to prevent performance problems leading to application outage:
Monitors write latency impact of GM and considers stopping a GM relationship or consistency group if impact exceeds a threshold
Link tolerance threshold allows ‘bursts’ of work to be tolerated
Defines period of time poor performance can be tolerated before GM stops
Settable from 1 minute to 1 day in 10 second increments (default 5 minutes)
Disabled by setting to ‘0’
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 14
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation27
Table of contents
Disk block virtualization
IBM SAN Volume Controller
Storage virtualization in enterprises today
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation28
Why To Use Storage Virtualization
� The management nightmare
Too many servers, operating systems, storage systems, management consoles, …
Too complex policies
Too large migration projects
� Availability requirements
� Storage resource utilization
� Storage performance
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 15
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation29
SANVolume Controller
SVC Version 4.1 Supported Environments
SAN with 4Gbps fabric
HitachiThunder
9200
95xxV
9520V
HPEVA
3000/5000
4000/6000
8000
IBMDS
DS4K / 6K
DS8000
DS4800
HitachiLightning
9980V
9970V
9910/9960
HPMA/EMA
8000
12000
16000
EMCSymm
8000
DMX
EMC/DellCLARiiON
FC4700
CX2/3/4/5/6/700
MicrosoftMSCS
MPIO, VSS, GDS
x64
IBMAIX
HACMP/XD
GPFS / VIO
SunSolaris 10VCS Clustering
SUN Cluster
HP/UX 11iV2TRU64
OpenVMSServiceGuard
Clustering
Linux(Intel/Power/zLinux)
RHEL/SUSE
W / LVM
RHEL AS 4
Ia32, x64, System p
IBMBladeCenterWin/Linux/VMWare/AIX
OPM/FCS/IBS
SAN
SANVolume Controller
Continuous Copy
Metro MirrorGlobal Mirror
VMWareWin / NW
guests
Point-in-time CopyFull volume
Copy on write
IBMESS,
FAStT
NovellNetWareClustering
Sun
9910/9960
9970/9980
HPXP
48, 128
512, 1024
. . .
1024Hosts
Cisco
McData
iSCSI to hostsVia Cisco IPS
HitachiTagma
StoreUSP, NSC55
IBMN series5200, 5500
NetAppFAS
3020, 3050
STKD173, 178, 220,
240, 280
FLX210, 240,
280, 380
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation30
Performance: SVC SPC-1 Benchmark
http://www.storageperformance.org/results/a00043-r1 _IBM_SPC1_executive-summary.pdf
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 16
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation31
Performance: SPC-2 Benchmark
� Newest addition to the Storage Performance Council Benchmarks
� Composite of three workloads to measure sequential performanceLarge file processing – scientific and large-scale financial processing
Large database queries – data mining and business intelligence
Video on demand – streaming movies to end users
� SVC posted the highest results in this industry-recognized storage performance testSVC (3.517 GB/s) + DS8300 (3.217 GB/s) lead the industry in this new benchmark
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation32
Enterprise Usage Today
� Widely accepted by customers:
Tape drive and library virtualization
Disk block level virtualization in LVM and storage subsystem
In-band file system virtualization (NAS)
Aggregated networks (iSCSI, long distance connections)
� Beginning of the lifecycle:
Disk block fabric-layer virtualization (in-band and out-of-band)
� Niche markets (this may change):
Virtual SANs
File and record virtualization
IT-Symposium 2007 17.04.2007
www.hp-user-society.de 17
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation33
Why Do Customers Hesitate?
� Reliability
SAN and storage failures may affect the whole datacenter.
SAN and storage failures may destroy data.
Security implications not clear.
� Complexity
Lack of standards.
Products difficult to understand and evaluate.
Multi-vendor support.
� Market situation
Religious wars about architectures.
Too many different solutions.
Storage Systems Division
Storage Virtualization with IBM SAN Volume Controller | 1G04 © 2007 IBM Corporation34
Disclaimer InformationCopyright © 2007 by Inte rnatio na l Bus iness Machines Corporation.
No part o f this document ma y be reprod uced or transmitted in any form w ithout writte n permissio n from IBM Corporation.
All stateme nts regarding IBM 's future dire ctio n and intent are subje ct to cha nge or withdra wa l without notice, and
represent goals and objective s only.
Refere nce s in this document to curre nt or future IBM produc ts, progra ms, or services does not imp ly tha t IBM intend s to make
suc h such products, programs or services availab le in a ll countr ies in whic h IBM operates or does bus iness.
THE INFORMATION PRO VIDED IN THIS DO CUMEN T IS DISTRIBUTED "AS IS" WITHOUT ANY WARRAN TY,
EITHER EXPRESS O R IMPLIED. IBM EXP RESSLY DISCLAIMS AN Y WARRANTIES O F MERCHAN TABILITY,
FITNESS FO R A PARTICU LAR PURPOS E OR IN FRIN GEMEN T. IBM shall ha ve no responsib ility to upda te this
info rmatio n. IBM products are warra nted accord ing to the terms and cond itions of the agree me nts (e.g., IBM Custo mer
Agree me nt, Statement of Limited Warra nty, Interna tional Progra m License Agreement, etc.) unde r whic h they are provided.
The provisio n o f the informatio n contained herein is no t intended to, and does not, gra nt any r ight or lice nse under any IBM
pate nts or copyrights. Inquir ies regarding patent or copyright licenses sho uld be made, in writing, to :
IBM D irector of Lice nsing
IBM Corporatio n
North Castle Drive
Armonk, NY 10504-1785
U.S.A.