Application Report

Migrating from Discrete

to Virtual Servers

Featuring

Development & QA Lab

OneConnect

HP ProLiant BL490c G7 Servers

Virtual Connect Flex-10

ESXi 5.0

Where IT perceptions are reality

Copyright 2012© IT Brand Pulse. All rights reserved. Document # APP2012001 v10 August, 2012

About

Company Overview Vonage (NYSE: VG) is a leading provider of communications services connecting individuals through broadband devices worldwide. With approximately 2.4 million subscribers, Vonage maintains and oper-ates a robust, high-quality, VoIP network that supports over six bil-lion communications per year. Through portable land-line phone re-placement and mobile calling applications, Vonage World and Vonage World Mobile plans offer unlimited calling to more than 60 countries for one low monthly rate. Vonage's service is sold on the web to customers in the U.S. (www.vonage.com), Canada (www.vonage.ca) and the United Kingdom (www.vonage.co.uk), and through regional and national retailers.

Development & QA Lab

Located at corporate headquarters in Holmdel, New Jersey, is the Vonage Development & QA Lab. It’s here that Vonage new technology and products are evaluated before they can enter a production environment. The Development & QA Lab also is where Vonage IT architects and engineers develop and certify the stand-ard hardware and software images which will be used in production systems ranging from front-end web servers to back-end database servers.

Migrating from Discrete to Virtualized Data Center

Taking advantage of technologies which will allow the company to consolidate infrastructure, reduce costs, and speed new server de-ployments, the Vonage data center in Holmdel is migrating from a mostly non-virtualized “discrete” environment, to a virtualized da-ta center. This installation features approximately 40 data center cabinets with 1,100 rack-mount servers — each dedicated to spe-cific applications. The power-hungry rack servers are connected to data center LANs and SANs by over 3,000 cables; deploying a new server takes about 2 weeks from the time a user makes a request.

Vonage is one of the largest VoIP providers

and an industry pioneer. Its name is a play on

their motto "Voice-Over-Net-AGE.”

Production servers in Holmdel, New Jersey

host a range of business-critical applications

ranging from front-end web servers to back-

end database servers.

Page 2 of 6 Document # APP2012001 v10, August, 2012

Reduce the space, power, cooling, and cabling needed by 1,100 rack-

mount servers. Shorten the time to deploy new application servers. Goals

Key Technologies

The combination of blade servers, server virtualiza-

tion and network virtualization, allow hundreds of

VMs to be deployed in a single cabinet.

Dense Compute Nodes

The Vonage migration from discrete data center to virtualized data center involves

many new technologies, products, and processes. Blade servers, server virtualiza-

tion and network virtualization stand out as technologies which are key to slashing

costs, while maintaining application performance and availability.

Blade Servers

Vonage is deploying HP ProLiant BL490c G7 blade servers to perform the same func-

tion as rack servers, but with far less floor space, power, cooling, and cables. By

sharing resources through a single enclosure, blade servers will also eliminate a

great deal of management complexity and overlap.

Server Virtualization

A typical application server utilizes only 1% to 5% of available CPU. Server virtualization al-

lows Vonage to fully utilize the compute power of each physical blade server by running mul-

tiple virtual machines (VMs) and applications. The result is another level of massive consoli-

dation and cost savings.

Network Virtualization

To accommodate the proliferation of VMs, the HP BladeSystems feature virtual networking capabilities which

allow server admins to configure unique virtual networks for each VM. Embedded on G7 servers used by

Vonage are dual-port integrated HP Virtual Connect 10Gb FlexFabric Adapters, which are based on Emulex

OneConnect® technology. A single HP 10Gb FlexFabric Adapter port can be configured as 4 FlexNICs, with an

option to configure one as a FlexHBA for either FC or iSCSI storage connectivity. Each FlexNIC or FlexHBA can

be configured with its own set of network policies tailored to the needs of specific VMs and applications.

One or two dual-port 10Gb HP Virtual Connect FlexFabric mez-

zanine cards can be added.

HP ProLiant BL490c G7 Server Blade

The dual-port 10Gb HP Virtual Connect FlexFabric Adapter based

on Emulex OneConnect technology is embedded on the server

blade motherboard and can be partitioned into 4 FlexNICs.

Page 3 of 6

A fully populated HP blade

system includes 16 server

blades, interconnected by a

single backplane, to LAN and

SAN switch fabrics embedded

in the chassis.

Document # APP2012001 v10, August, 2012

Killer App

Getting Started The 1,100 rack servers which form the Vonage discrete production environment are highly tuned for maximum performance and availability. The Vonage IT team was skeptical a mass migration to virtual servers could be accomplished without compromising one or both. Therefore the first step was to pilot the technology with a single blade system chassis and 2 server blades. It took about six weeks for Vonage architects to learn how to efficiently configure a physical server with VMware ESXi, and HP VirtualConnect Flex-10. Then, one application at a time, the number of VMs in the Development & QA lab grew to 900. The confidence of the IT team also grew as applications running in the lab demonstrated the perfor-mance needed in a production environment.

Live Migrations Emerge as a Killer App for On-board 10GbE Usage in the Development & QA Lab also demonstrated the

benefits of automated server performance optimization

and maintenance. With VMware Distributed Resource

Scheduler (DRS), scheduling server downtime to move ap-

plications to less utilized servers was no longer needed.

Without anyone knowing, DRS was automatically moving

VMs when servers became hot.

This made quick evacuation of VMs during vMotions a killer

application for the 10Gb HP Virtual Connect FlexFabric

Adapters onboard the G7 server blades. Vonage engineers

provisioned 4Gb FlexNICs to minimize the time required to

move hundreds of gigabytes of data involved in live migra-

tions, and quickly found the vMotions over-running the 4Gb

bandwidth.

Based on their experience in the lab, Vonage system architects decided to use 10Gb FlexNICs in production so

that ESXi has available the full bandwidth of the FlexFabric Adapter for live migrations.

A utility which allows ESXi admins to define the rules for manual or

automatic allocation of physical resources among VMs. DRS

Each of 1,100 rack servers

has its own power &

cabling system.

Maintaining application performance during live migrations

means moving hundreds of gigabytes of data as fast as possi-

ble — a killer application for the 10GbE FlexFabric adapters.

The migration project started with

a pilot using 2 server blades.

Page 4 of 6 Document # APP2012001 v10, August, 2012

Results & Lessons Learned

Mission Accomplished Today Vonage is implementing dense compute nodes consisting of blade servers, serv-

er virtualization, and network virtualization technology. As a result, the company will

consolidate 1,100 rack-mount servers, 40 cabinets and 3,000 cables into only 2 cabi-

nets, 4 blade server chassis, and a handful of cables. Along the way, Vonage architects

have identified a best practice for configuring I/O for live migrations, and a killer appli-

cation for 10GbE.

Lessons Learned The following are a few lessons the Vonage staff want to share with their peers:

Share your system designs with your vendors—Vonage architects discovered that

HP technical staff were aware of issues they encountered and would gladly have

reviewed their system design.

It takes a while to learn how the virtualization layers work together—It took six

weeks to learn how to use the server and network virtualization tools efficiently.

However, after learning proper configuration techniques, profiles can be cloned by

level 1 operators for rapid VM deployment.

One step at a time—18 months ago there was not a single VM in the Holmdel

Development & QA Lab. Starting with a couple of server blades, Vonage archi-

tects slowly added application workloads which allowed the rest of the IT team to

build confidence in the ability of VMs to perform.

vMotions can use an entire 10Gb Link—Testing showed that vMotions were sat-

urating the 4Gb FlexNICs provisioned in the lab.

Before virtualization—40 cabinets, 27.5 servers per cabinet. After virtual-

ization—2 cabinets, 2 blade server chassis per cabinet, 16 blade servers

per chassis, 17.2 VMs per blade server, 550 VMs per cabinet.

Results

Before—Each of 1,100 rack

servers has its own power

and cabling system.

After—Power and cabling

for one chassis supports 16

server blades and 550 VMs.

Page 5 of 6 Document # APP2012001 v10, August, 2012

Resources

Related Links To learn more about the companies, technologies, and products mentioned in this report, visit the following web pages: HP FlexFabric Adapters Provided by Emulex HP BladeSystem HP VirtualConnect Technology IT Brand Pulse VMware ESXi 5.0 VMware vSphere Distributed Resource Manager Vonage

About the Author

Frank Berry is founder and senior analyst for IT Brand Pulse, a trusted source of data and analysis about IT infrastructure, including servers, storage and networking. As for-mer vice president of product marketing and corporate marketing for QLogic, and vice president of worldwide marketing for the automated tape library (ATL) division of Quan-tum, Mr. Berry has over 30 years experience in the development and marketing of IT infrastructure. If you have any questions or comments about this report, contact frank.berry@itbrandpulse.com.

Page 6 of 6 Document # APP2012001 v10, August, 2012