Cluster Shared Volumes Reborn in Windows Server 2012

Gareth James Hani AdhamiSenior Consultant Senior Consultant

WSV423

Agenda

Overview of Cluster Shared Volumes (CSV)

What’s changed in Windows Server 2012

New CSV Architecture

Performance Enhancements

Improved Backup of Volumes

What are Cluster Shared Volumes (CSV)?

Clustered file system in Windows Server 2012

Layer of abstraction above NTFS

Simultaneous access to CSV volume by multiple cluster

nodes

LUN ownership abstracted from clustered application

Failover without drive ownership changes

No dismounting and remounting of volumes

Faster failover times (less downtime)

Addresses Management ComplexityChallenges of Managing Hyper-V Clusters

Manageability• Multi-path• Masking several LUN’s

Flexibility• LUN - smallest unit of

failover

Capacity• Poor SAN space

utilisation

Scalability• Complexity with drive

letters

Cluster Shared Volumes as you know it TodayWindows Server 2008 R2

First introduced in Windows Server 2008 R2

Only supported Hyper-V workloadFocused v1 release targeted at enabling Hyper-V

Implemented as file system mini-filter Intercepted and routed I/O

CSV Motivations for Windows Server 2012

Expand CSV to more workloads

File Server in addition to Hyper-V

Improve Backup

Improved performance

Direct I/O for more scenarios

Support for Spaces storage virtualisation

Multi-subnet support

New CSV ArchitectureWhat it delivers

Improved interoperability with file system mini-filter drivers

Better interoperability : Anti-virus software, Backup Software

Application consistent distributed backups

Removed external authentication dependencies

Improved performance and resiliency

New CSV Architecture Cont….

Support for memory mapped files

Allows volume encryption

BitLocker encrypted volumes

Integrated with new File System features

Support for Offloaded Data Transfer (ODX)

Spot-fixing integrated to do online correction

Under the hood

CSV Architecture

Cluster Shared Volumes (CSV)I/O Synchronisation Overview

Metadata

Shared LUN

Shared StorageVHD VHD VHD

Read/Write

Simultaneous read/write access on all Cluster Nodes

Server side metadata synchronisation

- Avoids I/O interruptions

When do Metadata Updates Occur?Virtual Machine

Creation/deletion

Power on/off

Mobility (live/storage migration)

Extending dynamic VHD

Renaming VHD

Backup - Snapshot creation

Key TakeawaysMetadata updates - small operations, infrequent for VMsParallel metadata updates - non-disruptive for applications

Disk

Volume Manager

NTFS

CSV File System Filter

VMShare

Server / SMB

Node 2

Disk

VMShare

MUP/RDBSS/

SMB

Node 1

VMShare

MUP/RDBSS/

SMB

Node 3

Direct I/O

CSV VolumeM

gr

CSV Proxy File System

Storage Connection Broken or not present

Coordination Node

CSV VolumeM

gr

CSV Proxy File System

CSV VolumeM

gr

CSV Proxy File System

CSVFS

SAN

LBFO/RDMA

Simplified SetupConfiguring a CSV Disk

Failover Cluster Manager Storage view integration

“Cluster Shared Volumes” container removed

CSV integrated into Failover Cluster core feature

No longer needs to be explicitly enabled

Simply right-click to add to CSV – that’s it!

demo

Setting up Clustered Shared Volumes

Single NamespaceConsistent view across the cluster

Single consistent file name spaceFiles have same name and path on any cluster node

Volumes exposed under “ClusterStorage” root directoryVolumeX directory name can be renamed

CSV NamespaceMount Points

Used custom reparse points in Win2008 R2

Win2012 uses standard Mount Points

Delivers better interoperability with:Performance Counters

System Center Operations Manager

Monitoring free space on CSV volumes

Better interoperability with backup software

CSV Proxy File System

CSV enabled volumes now appear as “CSVFS”

NTFS file system under the covers

Enables applications to be CSV aware

Ensures compatibility

Resiliency

How CSV Enables Even Higher Availability

Fault Tolerant Application HandlesCSV Resiliency

CSV provides I/O fault tolerance

Transparently handles node, network, and HBA failures

CSVFS virtualises file handles to applications

Volume Paused - I/O queued

Reopens “true” files handles and remaps the “virtual”

handles

Volume Resumed – I/O completed

Failover is transparent to application!

VHD

I/O Connectivity Fault Tolerance

VM running on Node 2 is unaffected

Coordination Node

SAN Connectivity

Failure

I/O Redirected via network

VM’s can then be live migrated to another node with zero client downtime

VHD

Node Fault Tolerance

Volume relocates to a healthy node

Brief queuing of I/O while volume

ownership is changed

Node Failure VM running

on Node 2 is

unaffected

Coordination Node

New Coordinator

Node

VHD

Network Fault Tolerance

Volume mounted on Node 1 Network Path

Connectivity Failure

Metadata Updates

Rerouted to redundant network

Fault-Tolerant TCP connections make a path failure seamless

VM running on Node 2 is

unaffected

demo

Cluster Shared Volume Resiliency

Performance

Improved CSV Performance

Cluster Shared Volumes CachingImproved CSV I/O Performance

Windows Cache Manager integration

Buffered read and write I/O’s cached like traditional NTFS

CSV Block Cache

Read-Only cache for un-buffered I/O

Distributed cache consistent across cluster

Huge value for Pooled VM VDI scenarios

512 MB recommended value

Disabled by default

No downtime to modify

demo

Cluster Shared Volume Block Cache in action

Redirected I/O Less OftenCSV Optimisations

Direct I/O for more scenarios

Delivers faster I/O performance and lower network overhead

Direct I/O for all types of file opens

Buffered Reads and Writes

Better VM creation and copy performance

New algorithm for I/O redirection detection

Opportunistic Locking as distributed locking mechanism

• Block level I/O performance parityDirect I/O• Remote file system (SMB) performance

parityRedirected I/O

Block Level I/O RedirectionHigh Performance fault condition I/O redirection

2x performance over File System

redirection

Disk

Volume Manager

NTFS

CSV File System Filter

VMShare

Server / SMB

Node 1

VMShare

MUP/RDBSS/

SMB

Node 2

Storage Connection Broken or not present

Coordination Node

CSV VolumeMg

r

CSV Proxy File

System

CSV VolumeMg

r

CSV Proxy File

System

CSVFS

LBFO/RDMA

Avoids traversing file system stack

twice

SMB 3.0 Performance ImprovementsInherit gains for CSV redirection performance

Improved Performance of refactored SMB 3.0 client (98%)

Network transport optimisationsTCP/IP – SMB multi-channel & NIC Teaming, TCP offloads, DC-TCP

RDMA – Lowest network CPU overhead (cycles/byte)

SMB SMB + DAS0

500

1000

1500

2000

2500

3000

3500

4000

4500

1175 (27%)

4270 (98%) 4315 (100%)

SQL TPC-C transactions/s

High Speed CSV I/O RedirectionSMB 3.0 integration

SMB multi-channelTraffic streamed across multiple networksImproved I/O performance in redirected mode

SMB Direct (SMB 3.0 over RDMA)

CSV Streaming I/O Across Multiple

Networks

Coordinator Node

VHD

10.10.10.X

20.20.20.X

VM with I/O being

redirected

Performance Improvements – RecapCSV Redirected mode enhancement summary

I/O redirection needed less

often

CSV Block Level Redirection

SMB multi-channelDirect (RDMA)

Improved SMB performance ~2% off block

Many Pieces Come Together for Radical Improvement

Deployment

CSV Deployment Considerations

CSV RequirementsThe requirements to enable a disk to be CSV

Basic diskFormatted with NTFS file system

FAT / FAT32 / ReFS not supported

CSV enabled disk cannot be used as a Witness Disk (aka. quorum disk)Can be basic or a Space

Simple SpaceMirrored Space with block level Redirected I/OParity Space not supported

Cannot have Windows Dedup enabled

Planning VM Density Per CSV VolumeDeployment considerations

How many VMs per CSV volume? No CSV volume restrictions VMFS limitations do not apply to CSV

CSV volume Metadata updates orchestrated server side and parallelised

How many IOPS can your storage array handle?

Backup

CSV Backup

CSV Backup Key Wins

Distributed Snapshots

• Distributed app consistent snapshot creation across cluster

Non-disruptive backups

• CSV volume ownership does not change during backup

Parallel Backups

• On same or different• CSV volumes• Cluster nodes

Improved Interoperability

• Backup applications / requestors not required to be ‘CSV aware’

• With filter driversImproved I/O performance

• Direct I/O mode for software snapshots

SummaryKey Takeaways

CSV significantly enhanced in Windows Server 2012Support for more workloadsHigh performanceName a concern, it’s gone!CSV is a core infrastructure to enable your private cloud

INFRASTRUCTUREa s a S E RV I C E

Related Content

Find Us Later At the ‘Speaker Lounge’ (12:45 – 13:45)

Breakout Sessions (session codes and titles)

VIR312 - What's New in Windows Server 2012 Hyper-V, Part 1

VIR315 - What's New in Windows Server 2012 Hyper-V, Part 2

VIR314 - WS2012 Hyper-V Live Migration and Live Storage Migration

WSV326 - The Path to Continuous Availability with WS2012

WSV332 - Cluster-Aware Updating and the New Generation of WSUS

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.