Copyright © 2008 VMware, Inc. All rights reserved.
Computing on a Distributed, Virtualized Infrastructure
Beng-Hong Lim
Senior Director of R&D
VMware
CERCS Distinguished Lecture and KeynoteGeorgia Tech, October 17, 2008
Copyright © 2008 VMware, Inc. All rights reserved.
Outline
Why virtualize?
How virtualization helps
Challenges and opportunities
Recap and speculate
Copyright © 2008 VMware, Inc. All rights reserved.
Why virtualize?
Common reason: share limited resources
past: expensive hardware
Today: hardware is cheap, but …
space, power, cooling
Copyright © 2008 VMware, Inc. All rights reserved.
BEFORE AFTER
Servers 1,000 80
Storage 270 TB DAS 140 TB SAN & NAS
Network 3,000 cables/ports 300 cables/ports
Facilities200 server racks
400 power whips
10 server racks
20 power whips
Customer Example: Leading North American Utility
Server consolidation
Copyright © 2008 VMware, Inc. All rights reserved.
Virtual Machines, Real Savings
80% reduction in space, power and cooling
$8 million saved over 2 years
Server consolidation
Copyright © 2008 VMware, Inc. All rights reserved.
Server consolidation is now mainstream
“Virtualization makes one computer act like many”- popular press tagline
Copyright © 2008 VMware, Inc. All rights reserved.
Beyond server consolidation
Management complexity
hardware: distributed, heterogeneous
software: distributed, complex
Distributed infrastructure management:the next killer app for virtualization
“Why is your part taking so long?
Copyright © 2008 VMware, Inc. All rights reserved.
BEFORE AFTER
Servers 1,000 80
Storage 270 TB DAS 140 TB SAN & NAS
Network 3,000 cables/ports 300 cables/ports
Facilities200 server racks
400 power whips
10 server racks
20 power whips
Customer Example: Leading North American Utility
Beyond server consolidation
Copyright © 2008 VMware, Inc. All rights reserved.
Operational efficiency
server and app load from 40 hrs to 30 mins
10,000 person-hours saved per year
Beyond server consolidation
Copyright © 2008 VMware, Inc. All rights reserved.
Workloads per Virtual Infrastructure Admin
50 workloads per admin for physical infrastructure admin
600
350
300
200
90
A Pharmaceutical Company
A Medical Center
An Investment Bank
A Large Consumer Bank
A Foreign Investment Bank
Copyright © 2008 VMware, Inc. All rights reserved.
Distributed Infrastructure Management
To better manage modern hardware and software
Virtualization:
liberates software from hardware
encapsulates software within versatile virtual execution environments
automates management of hardware as a giant pool of resources
Copyright © 2008 VMware, Inc. All rights reserved.
Liberating software from hardware
App-1
OS/1
HW-1
App-2
OS/2
HW-2
Copyright © 2008 VMware, Inc. All rights reserved.
Liberating software from hardware
App-1
OS/1
App-2
OS/2
VMM
HW-1
VMM
HW-2
Virtual Hardware Abstraction
Copyright © 2008 VMware, Inc. All rights reserved.
VMM VMM
Software as a highly-manageable object
Snapshot
Fork
Migrate
Record
Replay/Rewind
etc.
Log
Dis
kS
tora
ge
Ph
ysic
alM
ach
ines
Vir
tual
Mac
hin
es
Virtualization provides new ways to manipulate software
Copyright © 2008 VMware, Inc. All rights reserved.
Distributed Virtualization Layer
Exchange Server
Hardware as a distributed pool of resources
Allocate resources and capabilities on-demand to applications
Utilize redundant hardware for reliability and availability
CPU 2 x 1GHz
Memory 4 GB
Disk 500 GB
Network 1 Gbps
Fault Tolerant Yes
Disaster Recovery Enabled
Security High
Virtualization makes many computers act like one
Copyright © 2008 VMware, Inc. All rights reserved.
Distributed Infrastructure Management
To better manage modern hardware and software
How do we achieve this?
Copyright © 2008 VMware, Inc. All rights reserved.
Distributed hardware and software trend
High-speed connectivity
Web-based applications
Copyright © 2008 VMware, Inc. All rights reserved.
Server Management Virtual machines are hardware independent
VMotion (Live Migration)
Pre-copy memory
Virtual disks on networked storage
Preserve VM’s identity: name, MAC address, IP address
Copyright © 2008 VMware, Inc. All rights reserved.
Automatic load balancing across machines
Distributed ResourceScheduling (DRS)
Dynamic BalancingContinuous Optimization
Copyright © 2008 VMware, Inc. All rights reserved.
Adding and removing resources
Hot-plug machines
Add/remove capacity on demandImprove application availabilityDistributed power management
VMware VirtualCenter
X
Copyright © 2008 VMware, Inc. All rights reserved.
Power Savings with DPM
50% Savings During VMmark™
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Wat
ts
No DPM
With DPM
5 hour VMmark run
110 VMs, 4 server cluster
Copyright © 2008 VMware, Inc. All rights reserved.
Storage Management
Optimize disk usage, availability and management
Storage resource pool
Storage VMotion: live storage migration
Thin provisioning
Copyright © 2008 VMware, Inc. All rights reserved.
Storage VMotion
Migrate running VM to new storage
VM stays on same host
Virtual disks may be individually placed
Storage type independent
Migration does not disturb VM
No downtime
Transparent to guest OS and apps
Minimal performance impact
Copyright © 2008 VMware, Inc. All rights reserved.
Storage VMotion Benefits
Retire or migrate between arrays
Arrays coming off maintenance/leasing cycles
Storage tiering
Migrate from FC to iSCSI, NAS or within or between enclosure(s)
Eliminate performance bottlenecks
Load balance through LUN reconfiguration
Seamlessly add and begin using new LUNs
Non-disruptive VM file system upgrade
Future proofing disk format
Copyright © 2008 VMware, Inc. All rights reserved.
Thin Provisioning: Smart Allocation
VirtualDisks
PhysicalStorage
App
OS
App
OS
App
OS
VMware ESX
20GB 40GB 100GB
5GB20GB
40GB
5GB 40GB20 GB
Copyright © 2008 VMware, Inc. All rights reserved.
Software Management
Copyright © 2008 VMware, Inc. All rights reserved.
Dev
QA
Release Management
CCR
Operators
Ops
1 2 3Developers and QA
Traditional Software Lifecycle
develop test stage deploy manage integrate
(Change, Configuration and Release Management)
(Operations, Optimization and Support)
Different environments at each stage, hard to bridge
Copyright © 2008 VMware, Inc. All rights reserved.
Virtualization eases software development
VMware Workstation
Indispensible software development tool
Developer-oriented features
VM teams to model distributed hardware
Multiple snapshots
Scripted control of VM operations
Record/Replay
VAssert
VProbes (DTrace for VMs)
A rich set of tools not available in physical environments
.5.5
Copyright © 2008 VMware, Inc. All rights reserved.
Software Deployment and ManagementProcess Process Process
App Stack A App Stack B
DB2 9
App Stack C
Different application stacks have different configurations and requirements
Copyright © 2008 VMware, Inc. All rights reserved.
Checkpoint
Windows
Apache
Linux
Linux
Linux
FIREWALL
Tomcat
mySQLCRM
Virtual Appliance
Virtual Appliance
Virtual appliance as unit of deployment
Pre-built, pre-configured and ready-to-run software application packaged with the OS inside a Virtual Machine
Or packaged inside multiple Virtual Machines
Copyright © 2008 VMware, Inc. All rights reserved.
Virtual appliance packagingOpen Virtual Machine Format (OVF)
a vendor and platform independent standard for packaging and distributing virtual machines
handles single VMs and multi-tier VM “teams”
OVF 1.0 spec released by DMTF
The OVF descriptor file includes critical infrastructure information
describes virtual hardware configuration and properties
tells management tools and hypervisors what to do with the virtual machines
resource management, security, and availability become properties of the virtual machine, independent of where or how it is deployed
X.Ovf
Security = HighPerformance = 500 msAvailability = 99.99%
Access controlIP port restrictions
Apache
Linux
Linux
LinuxTomcat
mySQL
SLADefinitions
Virtual Appliance
Copyright © 2008 VMware, Inc. All rights reserved.
Dev, test, debug with virtual machines
Dev
QA
(Change, Configuration and Release Management)
CCR
(Operations, Optimization and Support)
Ops
develop test stage deploy manage integrate
1 2 3
SHARED VIRTUALINFRASTRUCTURE
Distributed, Multi-site developmentDev/Staging Process Automation
Deploy and Manage Complex ApplicationsAs “Enterprise Virtual Appliances”
Software Lifecycle on a Virtual Infrastructure
Copyright © 2008 VMware, Inc. All rights reserved.
Draw the Logical Datacenter and…
VI automatically deploys to physical resources and…
Virtual Infrastructure
InterconnectPool
CPUPool
MemoryPool
StoragePool
dynamically adapts the mappings to optimize efficiency within SLAs and policies
Copyright © 2008 VMware, Inc. All rights reserved.
Outline
Why virtualize?
How virtualization helps
Challenges and opportunities
Recap and speculate
Copyright © 2008 VMware, Inc. All rights reserved.
Challenge: supporting mobility
Uniform hardware abstraction
CPU: x86 standard
I/O devices: emulated vs. passthrough
Network switch management
port state, configuration and control
long-distance VMotion
Copyright © 2008 VMware, Inc. All rights reserved.
Passthrough I/O Devices
I/O MMU
VF VF VF
PF
PF = Physical Function, VF = Virtual Function
I/O Device
Guest OS
Device Driver
Guest OS
Device Driver
Guest OS
Device Driver
VirtualizationLayer
Basic passthrough support
I/O MMU for DMA isolation (Intel VT-d, AMD IOMMU)
Partitionable I/O devices for device sharing (PCI-SIG IOV)
However, we lose
uniform abstraction and VM mobility
Potential solutions:
paravirtualization
guest driver coordination
standardized hardware interfaces
Copyright © 2008 VMware, Inc. All rights reserved.
Network Switch Management Network-centric view:
virtualization blurs the host/network boundary
Virtual network switches and topologies within a host
VMs can move anywhere
The virtual network switch is a new access layer.
Network administrators can no longer rely on physical port-based access control
distribution
core
access
Extend switching and management protocols to virtual environments.
hosts
VMs
Copyright © 2008 VMware, Inc. All rights reserved.
Distributed Virtual Switches
vSwitch vSwitchvSwitch
Distributed Virtual Switch
Today’s vSwitch
Distributed vSwitch
Copyright © 2008 VMware, Inc. All rights reserved.
Migrates port state with VM
ESX host 1 ESX host 3ESX host 2
Copyright © 2008 VMware, Inc. All rights reserved.
Opportunity: making virtual better than real
Security
inspect OS activity from a safe platform, e.g., another VM.
Provide APIs for anti-virus software
Overshadow: defending applications from compromised OSes (Chen et al., ASPLOS 2008)
minimize and harden virtualization stack
Fault tolerance
software-based tolerance against hardware failures
run VMs in lockstep (Bressoud and Schneider, SOSP 1995)
Synthesize useful functions in virtual hardware
Copyright © 2008 VMware, Inc. All rights reserved.
Virtual Machine Fault Tolerance
X
Lockstep Lockstep
Primary Secondary SecondaryPrimary
X
Copyright © 2008 VMware, Inc. All rights reserved.
Virtual Machine Record & Replay
REPLAY
Application
Operating System
Virtualization Layer
Application
Operating System
Virtualization Layer
RECORD
Logging causes of non-determinism• Input (network, user), asynchronous I/O
(disk, devices), CPU timer interrupts
Deterministic delivery of events previously logged
• Result = repeatable VM execution
Copyright © 2008 VMware, Inc. All rights reserved.
Determinism
Given the exact same inputs, a processor will deterministically execute the same instruction stream and end up in the exact same state
“Input” is anything outside the CPU/memory that is visible to software:
I/O and interrupts
non-deterministic processor behavior (e.g., CPU timestamp)
Hard to do on physical machines, but doable for virtual machines
Record/replay in VMware Workstation for over a year
Copyright © 2008 VMware, Inc. All rights reserved.
Opportunity: client virtualization
Copyright © 2008 VMware, Inc. All rights reserved.
Client virtualization
User experience
access to applications and data from any device
with good graphics and interactive performance
on one easy to manage “desktop” environment
Centralized management
OS and application deployment and management
data backup and recovery
recovering from corruption
Thick vs. Thin, Mobile vs. Fixed, Online vs. Offline
Copyright © 2008 VMware, Inc. All rights reserved.
Client virtualization
A solution: virtual desktop in a VM
all the benefits of software lifecycle management
maintain single master version
Distributed virtual hardware: migrate computation and data to appropriate locations
CPU and memory: client vs. server
Graphics processing: local vs. remote
Virtual disk storage: local vs. networked, replicated, deduplicated
Copyright © 2008 VMware, Inc. All rights reserved.
Client virtualization scenario
Compute Graphics Storage
Online, standard graphics
Copyright © 2008 VMware, Inc. All rights reserved.
Client virtualization scenario
Compute Graphics Storage
Online, high-speed graphics
Copyright © 2008 VMware, Inc. All rights reserved.
Client virtualization scenario
Compute Graphics Storage
Offline
Copyright © 2008 VMware, Inc. All rights reserved.
Client virtualization
A solution: virtual desktop in a VM
all the benefits of software lifecycle management
Distributed virtual machine hardware: migrate computation and data to appropriate locations
CPU and memory: client vs. server
Graphics processing: local vs. remote
Virtual disk storage: local vs. networked, replicated, deduplicated
No lock-in to a particular configuration
Copyright © 2008 VMware, Inc. All rights reserved.
Outline
Why virtualize?
Infrastructure Management
Challenges and Opportunities
Recap and Speculate
Copyright © 2008 VMware, Inc. All rights reserved.
Recap
Why virtualize?
server consolidation
distributed infrastructure management
Infrastructure Management
Challenges and Opportunities
Recap and Speculate
Copyright © 2008 VMware, Inc. All rights reserved.
Recap
Why virtualize?
Infrastructure Management
data center management
software lifecycle management
Challenges and Opportunities
Recap and Speculate
Copyright © 2008 VMware, Inc. All rights reserved.
Recap
Why virtualize?
Infrastructure Management
Challenges and Opportunities
maintaining mobility
making virtual better than real
client virtualization
Recap and Speculate
Copyright © 2008 VMware, Inc. All rights reserved.
Speculating about the future
A Virtual, Mobile and Cloudy Future?
in your hand: multi-function mobile devices
in the cloud: multiple data centers
virtual appliances everywhere
Everything more distributed, more mobile
Copyright © 2008 VMware, Inc. All rights reserved.
Mobile clients
Proliferation of mobile client devices
notebooks, multi-function handhelds, smart phones
Can a mobile device serve all your computing needs?
limited size, compute power, network bandwidth
one (small) size does not fit all
We’ll be using various types of client devices
user environment and computation typically tied to device
User environment, data and computation should migrate automatically and seamlessly to appropriate hardware
Copyright © 2008 VMware, Inc. All rights reserved.
In the data center
Convergence of data center hardware
Traditional roles of data center hardware
Servers host applications
Storage arrays serve and manage storage blocks
Network switches process and route network traffic
Storage arrays and network switches are appliances
In a virtualized world
move storage and network processing into virtual appliances
servers = switches = storage = VM hosts
Copyright © 2008 VMware, Inc. All rights reserved.
In the cloud
Improved economics: shared, managed infrastructure
Lightweight entry and exit
Consumption-based pricing
Accessible using standard Internet protocols
Scalable and elastic
How Do We Define The Cloud?
Copyright © 2008 VMware, Inc. All rights reserved.
Challenges of Cloud Computing
Need for New, Highly Efficient andFlexible Computing Infrastructure
Multi-tenancy
Application Compatibility
Lack of Standardization CreatesComplexity and Switching Costs
Copyright © 2008 VMware, Inc. All rights reserved.
Cloud Services
Virtual Datacenter Off-Premise CloudsImage ManagementUser Accounts
ChargebackMobility
vCloud APIs
Copyright © 2008 VMware, Inc. All rights reserved.
PrimaryDatacenter
Cloud Provider
Federation: Connecting the Clouds
Secondary or BranchDatacenter
vApp
vApp
Copyright © 2008 VMware, Inc. All rights reserved.
Seamless Cloud Connectivity
vCloud Provider
1
Secondary or BranchDatacenter
3
3 SecurityvApp
PrimaryDatacenter
vApp
Copyright © 2008 VMware, Inc. All rights reserved.
A Distributed, Virtualized Infrastructure
Storage Network
Firewall Mail Web
Virtual Infrastructure
Copyright © 2008 VMware, Inc. All rights reserved.
In conclusion
Virtualization plays a central role in managing the globally-distributed infrastructure
run applications anywhere
manage software and hardware independently
break down traditional boundaries
Essential properties
uniform, isolated and mobile execution environment
global resource pools
compatibility
Do more with less
Copyright © 2008 VMware, Inc. All rights reserved.
Thank You!
Top Related