Multi-Site Clustering with Windows Server 2008 Enterprise

Symon PerrimanProgram ManagerMicrosoft CorporationWSV316

Multi-Site Clustering

Benefits Deployment Replication Networking Faster Failover Quorum Best Practices

Benefits of a Multi-Site ClusterProtects Against Loss of an Entire Datacenter

Power outage, fires, hurricanes, floods, earthquakes, terrorismAutomates Failover

Reduced downtimeLower complexity of disaster recovery plan

Reduces Administrative OverheadAutomatically synchronize application and cluster changesEasier to keep consistent than unclustered servers

What is the primary reason why disaster recovery solutions fail?

Dependence on People

Multi-Site Clustering Checklisthttp://technet.microsoft.com/en-us/library/dd197546.aspxOrganized multi-site cluster deployment guide

Multi-Site Clustering

Benefits Deployment Replication Networking Faster Failover Quorum Best Practices

Site B

Multi-Site Clustering Basics2+ physically separate sites1+ node at each siteStorage at each site with data replicationApplication moves during a failover

Site A

SANSAN

Redundancy Everywhere2 or more computers (nodes)2 NICs

3rd NIC for iSCSIHBA

Fibre Channel (FC)Serial Attached-SCSI (SAS)

Multipath IO (MPIO)Redundant Storage InterconnectsReplicated StorageOS, Service or Application HA Roles

Mix and Match Hardware You Can Use Any Hardware Configuration if

Each component has a Windows Server 2008 / R2 logoServers, Storage, HBAs, MPIO, etc…

It passes ValidateIt’s That Simple!

Connect your Windows Server 2008 / R2 logo’d hardwarePass every test in Validate

It is now supported!If you make a change, just run Validate again

Details: http://go.microsoft.com/fwlink/?LinkID=119949

FCCP Failover Cluster Configuration ProgramWindows Server 2008 / R2Buy validated solutions

“Validated by Microsoft Failover Cluster Configuration Program” Not required for Microsoft support, must be logo’dMore information: http://www.microsoft.com/windowsserver2008/en/us/clustering-program.aspx

Introduction to Multi-Site Clusteringdemo

Cluster Validation and ReplicationMulti-Site clusters are not required to pass the Storage tests to be supported

Validation guide and policy:http://go.microsoft.com/fwlink/?LinkID=119949

Multi-Site Clustering

Benefits Deployment Replication Networking Faster Failover Quorum Best Practices

Why is Replication Needed?Loss of a site won’t cause complete data lossData must exist on other site after a failoverDifferent storage needs than local clusters

Multiple storage arrays, independent on each siteNodes usually access local site’s storage first

Site AChanges are made on Site A

and replicated to Site B

Site B

Replica

Replication Solutions

Replication LevelsHardware (block level) storage-based replicationSoftware (file system level) host-based replicationApplication-based replication

Exchange Server 2007 CCR

Replication TypesSynchronousAsynchronous

A data replication mechanism between sites is needed

Synchronous ReplicationHost receives “write complete” response from the storage after the data is successfully written on both storage devices

PrimaryStorage

SecondaryStorage

WriteComplete

Replication

Acknowledgement

WriteRequest

Asynchronous ReplicationHost receives “write complete” response from the storage after the data is successfully written to the primary storage device

PrimaryStorage

SecondaryStorage

WriteComplete

Replication

WriteRequest

Synchronous vs. Asynchronous

Synchronous AsynchronousNo data loss Potential data loss on

hard failuresRequires high

bandwidth/low latency connection

Enough bandwidth to keep up with data replication

Stretches over shorter distances

Stretches over longer distances

Write latencies impact application performance

No significant impact on application performance

What About DFS-Replication?

DFS-R performs replication on file close

Some file types stay open for a very long timeVHDs for Virtual MachinesDatabases for SQL Server

Data could be lost during a failover if it had not yet replicated

Using DFS-R to replicate the cluster disk’s datain a multi-site Failover Cluster is not supported

Disk Resource

Resource Group

Custom Resource(manages replication)

IP Address Resources*

Network Name Resource

Establishes start order

timing

Group determines

smallest unit of failover

Resource Dependencies

Workload Resource (example File Server)

“ depends on ”

Multi-Site Clustering

Benefits Deployment Replication Networking Faster Failover Quorum Best Practices

Site B

Network Considerations

Cluster nodes can reside in different subnets (2008/R2)No need to connect nodes with VLANs

Site A10.10.10.1 20.20.20.1

30.30.30.140.40.40.1

Public Network

Separate Network

Stretching the NetworkLonger distance means greater network latencyToo many missed health checks can cause false failoverFully configurable in 2008/R2

Failover Clustering has NO DISTANCE & NO SUBNET LIMITATIONSCheck if your vendor’s hardware / replication has limitations

SameSubnetDelay (default = 1 second)Frequency heartbeats are sent

SameSubnetThreshold (default = 5 heartbeats)Missed heartbeats before an interface is considered down

CrossSubnetDelay (default = 1 second)Frequency heartbeats are sent to nodes on dissimilar subnets

CrossSubnetThreshold (default = 5 heartbeats)Missed heartbeats before an interface is considered down to nodes on dissimilar subnets

Command Line: Cluster.exe /propPowerShell (R2): Get-Cluster | fl *

Security Over the WANImproved SecurityPrevent Clients from Connecting to NetworksEncrypt Intra-cluster Traffic

0 = clear text1 = signed (default)2 = encrypted

IP AddressResource B

IP AddressResource A

Enhanced Dependencies – OR

Network Name resource stays up if either IP Address Resource A OR IP Address Resource B is up

Network Name Resource

OR

Disk Resource

Workload Resource (example File Server)

IP Address Resources A

Network Name Resource

Resource Dependencies

IP Address Resources B

Comes online on site A

Comes online on site B

OR

Custom App(replication)

Multi-Site Clustering

Benefits Deployment Replication Networking Faster Failover Quorum Best Practices

DNS UpdatesNodes in dissimilar subnetsFailover changes resource’s IP AddressClients need that new IP Address from DNS to reconnect

10.10.10.111 20.20.20.222

DNS Server 1 DNS Server 2

DNS Replication

Record Updated

Record Created

Record Obtained

FS = 10.10.10.111

Record Updated

FS = 20.20.20.222Site A Site B

Network Name PropertiesRegisterAllProvidersIP (default = 0 for FALSE)

Determines if all IP Addresses for a Network Name will be registered by DNSTRUE (1): IP Addresses can be online or offline and will still be registeredEnsure application is set to try all IP Addresses, so clients can come online quicker

HostRecordTTL (default = 1200 seconds)Controls time the DNS record lives on client for a cluster network nameShorter TTL: DNS records for clients updated soonerExchange Server 2007 recommends a value of five minutes (300 seconds)

Local Failover FirstLocal failover first

No change in IP AddressCross-site failover for disaster recovery

10.10.10.111

DNS Server 1 DNS Server 2

FS = 10.10.10.111Site A Site B

20.20.20.222

FS = 20.20.20.222

Failover OrderPreferred Owners

Local failover first

Possible Owners Always EnforcedResource will not start on non-possible owner

AntiAffinityClassNames Groups with same AACN try to avoid moving to same nodehttp://msdn.microsoft.com/en-us/library/aa369651(VS.85).aspx

Virtual LAN (VLAN)Deploying a VLAN minimizes client reconnection times

Can be harder to configure

Required for SQL & live migration

10.10.10.111

DNS Server 1 DNS Server 2

FS = 10.10.10.111

Site A Site B

VLAN

10.10.10.111

Multi-Site Clustering Groups and Settingsdemo

Multi-Site Clustering

Benefits Deployment Replication Networking Faster Failover Quorum Best Practices

Quorum Overview

Node majorityNode and File Share majority

Disk only (not recommended)Node and Disk majority

Vote Vote Vote VoteVote

Majority is greater than 50%Possible Voters:

Nodes (1 each), Disk Witness (1 max), File Share Witness (1 max)4 Quorum Types

Node and Disk MajorityNodes get 1 vote each and Disk gets vote

Loss of disk or node OK if majority is maintainedDo not use in multi-site clusters unless directed by vendor

Vote VoteVote

Replicated Storage from vendor

?

Node Majority

Site BSite A

Cross site network connectivity broken!

Can I communicate with majority of the nodes in the cluster?

Yes, then Stay Up

Can I communicate with majority of the nodes in the cluster?

No, drop out of Cluster Membership

5 Node Cluster: Majority = 3

SAN SAN

Majority in Primary Site

Node Majority

Site BSite A

Disaster at Site 1

We are down!Can I communicate with majority of the nodes in the cluster?

No, drop out of Cluster Membership

SAN SAN

Majority in Primary Site

5 Node Cluster: Majority = 3

Forcing Quorum

Always understand why quorum was lostUsed to bring cluster online without quorumCluster starts in a special “forced” stateOnce majority achieved, no more “forced” state

Command line:net start clussvc /forcequorum (or /fq)

PowerShell (R2):Start-ClusterNode –FixQuorum (or –fq)

Site A

Multi-Site With File Share Witness

Site BWAN

Site C

SAN SAN

\\Foo\Cluster1Complete resiliency and automatic recovery from the loss of any 1 site

File Share Witness

Replicated Storage from vendor

WANSite A

Multi-Site With File Share Witness

Site B

Site C

SAN SAN

Complete resiliency and automatic recovery from the loss of any 1 site

File Share Witness

Replicated Storage from vendor

\\Foo\Cluster1

WANSite A

Multi-Site With File Share Witness

Site B

Site C

SAN SAN

Complete resiliency and automatic recovery from the loss of the File Share Witness

File Share Witness

Replicated Storage from vendor

\\Foo\Cluster1

FSW Considerations

Simple Windows File ServerNeeds to be in the same forestRunning Windows Server® 2003, 2008 or 2008 R2

Recommended to be at 3rd separate siteSingle file server can serve as a witness for multiple clusters

Each cluster requires its own shareCan be clustered in a second cluster

FSW cannot be on a node in the same clusterIt is an additional voter for free (almost)

Quorum on a Multi-Site Clusterdemo

Quorum Model Summary

No Majority: Disk OnlyNote RecommendedOnly use as directed by vendor

Node and Disk MajorityOnly use as directed by vendor

Node MajorityOdd number of nodes

Node and File Share MajorityBest availability solutionRecommended for

Exchange Server 2007 CCR

Multi-Site Clustering

Benefits Deployment Replication Networking Faster Failover Quorum Best Practices

Cluster your Branch OfficesCluster several standalone File Servers from branch officesKeep network traffic lowHigh-Availability for the filesRedundancy for the data

Site BSite A

Clients primarily accessing

applications in Site A

Clients primarily accessing

applications in Site B

Multi-Site Across the EnterpriseMore distributed cluster nodes & clusters gives higher availabilityComplete resiliency and automatic failoverRemember your quorum model

Loss of any single site should not bring down the clusterFile Share Witness

1 File Server hosts all File Share Witnesses for multiple clustersMake it highly-available

Separate siteNot a node in that same cluster

Cluster 1, Site 1

Cluster 2, Branch 1

Cluster 2, Main Office

Cluster 2, Branch 2

Cluster 1, Site 2 Cluster 3, Many FSWs

WANSite A

Multi-Site Clustering Review

Site B

Site C

SAN SAN

4, 6, 8… nodes + FSW = odd # votesLocal failover first (preferred owner)Site failover second (possible owner)AntiAffinityClassNames

File Share Witness

Replicated Storage from vendor

Faster DNS UpdatesRegister all IPs for a Network NameShorten client’s DNS record TTLEnsure application tries all IPs

Encrypt WAN traffic for securityAdjust health checks for latency

Configure ‘OR’ dependencies

Session SummaryMulti-Site Failover Clustering has many benefitsVariety of hardware options & configurationsRedundancy is needed everywhereUnderstand your replication needsCompare VLANs with multiple subnetsPlan your quorum model & nodes before deploymentFollow the checklist and best practices

http://technet.microsoft.com/en-us/library/dd197546.aspx

Are You Up For a Challenge?

Become a Cluster MVP!

Contact: ClusMVP@microsoft.com

Passion for High Availability?

question & answer

www.microsoft.com/teched

Sessions On-Demand & Community

http://microsoft.com/technet

Resources for IT Professionals

http://microsoft.com/msdn

Resources for Developers

www.microsoft.com/learningMicrosoft Certification and Training Resources

www.microsoft.com/learning

Microsoft Certification & Training Resources

Resources

Related ContentBreakout SessionsWSV310 Failover Clustering Feature Roadmap for Windows Server 2008 R2WSV313 Innovating High Availability with Cluster Shared Volumes (CSV)WSV316 Multi-Site Clustering with Windows Server 2008 EnterpriseVIR311 From Zero to Live Migration. How to Set Up a Live MigrationDAT302 All You Need to Know about Microsoft SQL Server 2008 Failover ClustersDAT306 Building a HA Strategy for Your Enterprise Using Microsoft SQL Server 2008 DAT322 Tips and Tricks for Successful Database Mirroring Deployments with Microsoft SQL ServerWSV311 High Availability and Disaster Recovery Considerations for Hyper-VWSV315 Implementing Hyper-V on Clusters (High Availability)UNC313 High Availability in Microsoft Exchange Server "14"UNC402 Microsoft Exchange Server 2007 HA and Disaster Recovery Deep DiveBOF52 Microsoft Exchange Server 2007 HA and Disaster Recovery: Are You Prepared?

Interactive SessionsWSV01-INT Failover Clustering Unleashed with Windows Server 2008 R2UNC02-INT Designing Microsoft Exchange Server "14" High Availability Solutions

Hands on LabsWSV16-HOL Windows Server 2008 R2: Failover ClusteringVIR03-HOL Implementing Windows Server 2008 Hyper-V HA and Quick MigrationDAT12-HOL Microsoft SQL Server 2008 Database Mirroring, Part 1DAT13-HOL Microsoft SQL Server 2008 Database Mirroring, Part 2UNC12-HOL Microsoft Exchange Server "14" High Availability and Storage Scenarios

Track ResourcesCluster Team Blog: http://blogs.msdn.com/clustering/ Cluster Information Portal: http://www.microsoft.com/windowsserver2008/en/us/clustering-home.aspx Clustering Technical Resources: http://www.microsoft.com/windowsserver2008/en/us/clustering-resources.aspx Clustering Forum (2008): http://forums.technet.microsoft.com/en-US/winserverClustering/threads/Clustering Forum (2008 R2):

http://social.technet.microsoft.com/Forums/en-US/windowsserver2008r2highavailability/threads/ Clustering Newsgroup:

http://www.microsoft.com/communities/newsgroups/list/en-us/default.aspx?dg=microsoft.public.windows.server.clustering

Failover Clustering Deployment Guide: http://technet.microsoft.com/en-us/library/dd197477.aspx TechNet: Configure a Service or Application for High Availability: http://technet.microsoft.com/en-us/library/cc732478.aspx TechNet: Installing a Failover Cluster: http://technet.microsoft.com/en-us/library/cc772178.aspx TechNet: Creating a Failover Cluster: http://technet.microsoft.com/en-us/library/cc755009.aspxWebcast (2008 R2): Introduction to Failover Clustering: http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032407190&Culture=en-USWebcast (2008 R2): HA Basics with Hyper-V: http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032407222&Culture=en-US Webcast (2008 R2): Cluster Shared Volumes (CSV):

http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032407238&Culture=en-US

Windows Server ResourcesMake sure you pick up your copy of Windows Server 2008 R2 RC from the Materials Distribution Counter

Learn More about Windows Server 2008 R2: www.microsoft.com/WindowsServer2008R2

Technical Learning Center (Orange Section): Highlighting Windows Server 2008 and R2 technologies•Over 15 booths and experts from Microsoft and our partners

Complete an evaluation on CommNet and enter to win!

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,

IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.