Brian Summers – Manager Datacenter...

watsonwyatt.com

Watson Wyatt World Wide

Technology & Administration Solutions (TAS)

Brian Summers –

Manager Datacenter

Operations

Copyright © Watson Wyatt Worldwide. All rights reserved

About Watson Wyatt

107 OFFICES WORLDWIDE

7,700 associates

in

33 countries

Serving more than

70% of the FORTUNE 500

More than $1.76 billion in 2008 revenue

Three Core Service Areas

TECHNOLOGY &ADMINISTRATION

SOLUTIONS

HUMAN CAPITAL

BENEFITS

A Top, Global Provider


About Watson Wyatt, continued

Company Wide:

1,500 Combined Servers

700 Virtual Machines

6 Datacenters

TAS

570 Physical Servers

350 Virtual Machines

2 Datacenters –

Virginia and California

2 Call Centers


HR Operations & Technology

Pension Administration

H & W Administration

Compensation Administration

Employee Communication

Employee Contact Centers

Employee Portal AccessConsulting Services

Focused on optimizing:• HR Operations• HR Technology• Employee Contact Centers• Vendor Management• Employee Portals

Broad Capabilities• Strategy/Design• Operational Audit• Benchmarking• Vendor Audit• Implementation Support

BenefitConnect PensionTM

Fully featured and cost-effective pension administration solution

High tech and high touch for:

• Retirement & termination events

• Benefit estimates• Indicative data

changes• General questions

Broad functionality• For employees• For administrators• For employee

contact centers

BenefitConnect HealthTM

Connecting benefits administration and consumer education

Three channels• Educate

employees• Evaluate options• Elect plans that fit

your needs

Broad functionality• For employees• For administrators• For employee

contact centers

RewardTM

Complete compensation automation

Complete solution• Survey data• Market pricing• Create structures• Model costing• Job leveling• Manager decision

support & budget control

Broad functionality• For administrators• For managers

eStatementsTM

Real time access to pay and benefits information

Communicate rewards value

• Integrates all providers

• Personalized content

• Interactive modeling tools

• Single sign-on

Broad functionality• For employees

Answer KeyTM

The technology you need to operate your HR contact center

Enterprise wide• Case

management• Workflow• Fulfillment• Employee data• Reporting

Broad functionality• For contact

centers• For employees• For HR

Advisory Services Plus Administrative Solutions to Deliver Benefits Administration, HR Operations and Employee Communication

Watson Wyatt’s Technology & Administration Solutions Practice


CLIENT RETAINED SERVICE

Solutions are configured to meet our clients’

specific needs whether they include...

...System

WATSON WYATT OUTSOURCED SERVICE

...System and Administration

...System, Administration and Customer Service Representatives

Flexible Benefits Administration Model


TAS Architecture


88

BenefitConnect™ Employee Self Service


TAS Architecture


Merging Datacenters –

The Challenge (2003)

High Utilization 3 MOS out of the yearHardware EfficiencyStaffing RequirementsDisaster RecoveryRapid DeploymentsSystem Availability (Maintenance Windows)


2005 2006 2007

Pre Virtualization:Hardware•Environmental

ESX 3.0 Occupy NDC

Conventional OpsMigrations

InfrastructureLimited Prod

Realize SavingsOpen Enrollment

VM Acceptance

Capital InvestmentHW InefficienciesResistance

Capital InvestmentHW Inefficiencies

Early VMware Timeline


2005 2006 2007

Pre Virtualization:Hardware•Environmental

ESX 3.0 Occupy NDC

Conventional OpsMigrations

InfrastructureLimited Prod

Realize SavingsOpen Enrollment

VM Acceptance

Capital InvestmentHW InefficienciesResistance

Capital InvestmentHW Inefficiencies

Early VMware Timeline

PERFORMANCEISSUES!


2007 Open Enrollment VMware Findings

IOPS and SCSI Reservations

Application paging and logging impact, metadata changes

Multiple ESX hosts hitting same LUN and SAN queuing requests at LUN

Inappropriate testing, application and correspondence to VM environment


Distributed Storage Option

SAN 1 SAN 2


Tiered Storage

Host 1FC

Host 2FC

Host 3FC

SharedNFS

Tier 1 Fiber Channel Tier 2 NFS


Lessons Learned: Validate Performance

In 2007 we ran out of IOPS and performance suffered

In 2008, we Validated Performance and implemented an appropriate architecture

A drop in throughput per VM with Random i/o revealed being disk bound at expected load

November 2008 Open Enrollment season went really well

"It's borderline on the simulator, we need to do more tests." --

Scotty

50% read 100% Random 4K reads/writes 4 workers/CPU

RequiredPerformance


Key Virtualization Benefits Benefits To Watson Wyatt:

–

HW Efficiencies–

Staff Efficiencies–

Easier DR Plan–

No New Datacenter–

Increased uptime–

Storage savings with Deduplication

VMware Calculator Benefits 3000 Sq Feet DC Space $300K Power Cooling $1.3 M Server Costs $100K networking ($200K) Storage $2.5M Total Savings

Physical

VMware

“We saved enough by virtualizing

to be ableto self fund disaster recovery in a

challenging economy”


Current TAS Environment

5,000,000 Participants300 Clients80 VLAN’s30 SQL Server270 VM’s400 VM’s

to be

Cisco 6500sBrocade 4 GB Fiber

HDS USP/VM for SQL Server

NTAP/IBM SANFC/NFS

HDS FC IBM/NTAP FC/NFS


Current TAS Environment, Networking

• VMware Switches at Distribution layer• OSPF and Rapid Spanning Tree


Future: Disaster Recovery

Physical Servers recovered Virtual

Use NetApp’s

Snapmirror

Script based Recovery

Add SRM when it supports NFS

Can Test without impacting Production

RTO 30 Minutes

Deduplicate

VM’s

before replication

DR Scout to replicate SQL writes to

SnapMirrorHerndon, VA San Diego, CA

NFS-NAS & FC-SAN NFS-NAS & FC-SAN

Vizioncore

+

DR Scout


Future: SAN Based Backup

No Backup Agents

‘Instant’

Backup

Application Consistent

Deduplication

–

Keep many backup’s

GUI for easy restore

Integrated with VMware via API’s

VM VM

Storage Pool

Follow advice on NetApp

TR3428 :VMware /NetApp

Best Practices

ESX 3.5U3 or later“Backups are quick because the data never moves”

vCenter


VMW ESXVMW ESXVMW ESX

Server 2Server 2

VMW ESXVMW ESXVMW ESX

Server 1Server 1

VM #5VM VM #5#5

VM #8VM VM #8#8

VM #7VM VM #7#7

VM #6VM VM #6#6

VM #4VM VM #4#4

VM #3VM VM #3#3

VM #2VM VM #2#2

VM #1VM VM #1#1

VMware vSwitchVMware VMware vSwitchvSwitch VMware vSwitchVMware VMware vSwitchvSwitchNexus 1000VNexus 1000VNexus 1000VNexus 1000VNexus 1000VNexus 1000V Nexus 1000V DVSNexus 1000V DVSNexus 1000V DVS

VM #8VM VM #8#8

VM #7VM VM #7#7

VM #6VM VM #6#6

VM #4VM VM #4#4

VM #3VM VM #3#3

VM #2VM VM #2#2

VM #5VM VM #5#5

VM #1VM VM #1#1

Upgrade Plan:

Start with an Empty ClusterMigrate Unimportant VM’sImplement Nexus 1000VTest Fault Tolerance Test SQL Server PerformanceUpgrade existing farms after OpenEnrollment

2009.

vSphere Upgrade Plan


23

Thank You

www.watsonwyatt.com

[email protected]

Brian Summers – Manager Datacenter...

Documents

Transcript of Brian Summers – Manager Datacenter...