Usenix Invited Talk

41
© 2001 VMware, Inc. All rights reserved. The Future of Virtual Machines: A VMware Perspective Ed Bugnion Co-founder, VMware Inc. JUGS September 27, 2001

Transcript of Usenix Invited Talk

Page 1: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.

The Future of Virtual Machines:A VMware Perspective

Ed BugnionCo-founder, VMware Inc.

JUGSSeptember 27, 2001

Page 2: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.2

Outline

• Historical Perspective

• MultipleWorlds™ Technology• Technology and Products

• Technology• Hosted and Host-less architectures• Performance

• 4 Usage scenarios

Page 3: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.3

The Problem (1960’s)

Mainframe Hardware

Operating System

Page 4: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.4

The Solution (1960’s)

Mainframe Hardware

Operating System

Operating System

Mainframe Hardware

Page 5: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.5

Virtual Machine Monitors

A thin software layer that sits between hardware and the operating system— virtualizing

and managing all hardware resources

IBM Mainframe

IBM VM/370

CMS MVS CMS CMS

App App App App

Page 6: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.6

Old idea from the 1960s

• IBM VM/370 – A VMM for IBM mainframe• Multiple OS environments on expensive hardware• Desirable when few machine around

• Popular research idea in 1960s and 1970s• Entire conferences on virtual machine monitor• Hardware/VMM/OS designed together

• Interest died out in the 1980s and 1990s.• Hardware got cheap• Operating systems got more more powerful (e.g multi-user)

Page 7: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.7

A return to Virtual Machines

• Disco: Stanford research project (1996-):• Run commodity OSes on scalable multiprocessors• Focus on high-end: NUMA, MIPS, IRIX

• Hardware has changed:• Cheap, diverse, graphical user interface• Designed without virtualization in mind

• System Software has changed: • Extremely complex• Advanced networking protocols• But even today :

•Not always multi-user•With limitations, incompatibilities, …

Page 8: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.8

The Problem Today

Intel Architecture

Operating System

Page 9: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.9

The VMware Solution

Intel Architecture

Operating System

Operating System

Intel Architecture

Page 10: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.10

VMware™ MultipleWorlds™ Technology

A thin software layer that sits between Intel hardware and the operating system—

virtualizing and managing all hardware resources

Intel Architecture

VMware MultipleWorlds

Win2000

WinNT Linux Win

2000

App App App App

Page 11: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.11

MultipleWorlds Technology

A world is an application execution environment with its own operating system

World

Intel Architecture

VMware MultipleWorlds

Win2000

WinNT Linux Win

2000

App App App App

Page 12: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.12

MultipleWorlds Technology

A world is an application execution environment with its own operating system

World

Intel Architecture

VMware MultipleWorlds

Win2000

WinNT Linux Win

2000

App App App App

Page 13: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.13

Challenges

• Virtualization of IA-32

• Hardware Diversity

• Acceptance

Page 14: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.14

VMware Workstation– Screen shot

Page 15: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.15

VMware Server – Screen Shot

• Web-based management interface• Stop, start, suspend/resume virtual

machines• Monitor CPU usage• Run scripts• Secure user authentication

• Remote Console• Windows and Linux versions• Full desktop display• Full mouse and keyboard support• Secure user authentication• Access VMware configuration editor

Page 16: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.16

VMware Products

• VMware Workstation • Run Multiple Operating Systems on your workstation• Hosted Architecture• Available for Linux and Windows hosts

• VMware GSX Server• Run multiple servers on your server• Hosted Architecture• Available for Linux hosts and soon Windows hosts

• VMware ESX Server• + Quality of Service• + High-performance I/O• Host-less Architecture

Page 17: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.17

Virtual Hardware

Floppy Disks

Parallel Ports Serial/Com Ports

Ethernet

Keyboard

Mouse

Monitor(VMM)

IDE Controller SCSI Controller

Sound Card

Page 18: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.18

Attributes of MultipleWorlds Technology

• Software compatibility• Runs pretty much all software

• Low overheads/High performance• Near “raw” machine performance

• Complete isolation• Total data isolation between virtual machines

• Encapsulation• Virtual machines are not tied to physical machines

• Resource management

Page 19: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.

VMware Core Technology

The present

Page 20: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.20

0VMM

Virtualization through Ring Compression

12

3 userVirtual Machine Monitor (VMM) runs at ring 0

Kernel(s) run at ring 1

Requires that CPUis virtualizable

kernel

Page 21: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.21

Classification of processor architectures

• Strictly virtualizable processor architectures• Can build a VMM based on trap emulation exclusively

•No software running inside the VM cannot determine the presence of the VMM (short of timing attacks)

• Examples: IBM S/390, DEC Compaq Intel Alpha, PowerPC

• (Non-strictly) virtualizable processor architectures• Trap emulation alone is not sufficient and/or not complete

•E.g. instructions have different semantics at various levels (sufficient)•E.g Some software sequences can determine the presence of the VMM (complete)

• Examples: IA-32, IA-64

• Non virtualizable processor architectures• Basic component missing (e.g. MMU, …)

Page 22: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.22

Hosted VMware Architecture

VMware achieves both near-native execution speed and broad device support by transparently switching* between Host Mode and VMM Mode.

Guest OS Applications

Guest Operating System

Host OS Apps

Host OS

PC HardwareDisks Memory CPUNIC

VMware App Virtual Machine

VMware Driver Virtual Machine Monitor

The VMware Virtual machine monitor allows each guest OS to directly access the processor (direct execution)

VMware, acting as an application, uses the host to access other devices such as the hard disk, floppy, or network card

VMM ModeHost Mode

*VMware typically switches modes 1000 times per second

Page 23: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.23

Hosted VMM Architecture

• Advantages:• Installs and runs like an application• Portable – host OS does I/O access• Coexists with applications running on the host

• Limits:• Subject to Host OS:

•Scheduling Decisions•Resource management decisions•OS failures

• Performance overheads:•World Switch• I/O access

• Usenix 2001 paper: J. Sugerman, G. Venkitachalam and B.-H. Lim, “Virtualizing I/O on VMware

Workstation’s Hosted Architecture”.

Page 24: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.24

Virtualizing a Network Interface

Host OS

PC HardwarePhysical NIC

VMApp

VMDriver

Guest OS

VMM

Phy

sica

l Eth

erne

t

NIC Driver

NIC Driver

Virtual Bridge

Virtual Network Hub

Page 25: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.25

Experiment – TCP Throughput

• Two speed of host:• Standard -- 733 MHz Pentium III• Slower -- 350 MHz Pentium II

• 100 megabit Ethernet connected via crossover cable

• Host and Guest OSes are Linux 2.2.x kernels

• 3 optimizations that reduce number of World switches

VMTCP

Host TCP Host Host Host

Native Virtual Machine

Page 26: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.26

Optimized Performance– 733 MHz

Native

VM/733 MHzVersion 2.0

VM/733 MHzOptimized

Page 27: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.27

Optimized Performance– 350MHz

Native

VM/350 MHzVersion 2.0

VM/350 MHzOptimized

Page 28: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.28

CPU Utilization – VM/PC-733

0

20

40

60

80

100

120

140

Version 2.0VMM I/O PortsVMM I/O Ports + Send CombiningVMM I/O Ports + Send Combining + IRQ Notification

Per

cen

t

•Native PC-733 is I/O bound with under 20% CPU utilization

Page 29: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.29

Beyond the Hosted Architecture

• Limits of the Hosted Architecture:• World switch overhead – especially I/O• Hard to make QoS guarantees• Depend on the Host

• ESX Server Architecture:• Eliminate the host• All applications run in a VM• Looks closer to a traditional VMM system

Page 30: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.30

ESX Server Architecture

Memorynic

nicNICdiskCPU

x86 SMPHardware

ConsoleOS

VMM

GuestOS

GuestOS

GuestOS

GuestOS

VMkernelScheduler

MemoryMgmt

SCSIDriver

EthernetDriver

VMMVMMVMM

Page 31: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.31

x86 SMPHardware

High Performance Network

VMwareServerVMM

NIC

Stub Driver

Shared Device

NIC specific drivers

•Ethernet and Gigabit Ethernet• Each virtual adapter has its own MAC address• No world switch !

VMware Ethernet Driver

NIC

Exclusive Device

VMM VMM

VMware Ethernet Driver

VMM

Stub Driver

Stub Driver

Page 32: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.32

x86 SMPHardware

Intra-system networking

VMwareServerVMM

Stub Driver

Stub Driver

Stub Driver

NIC specific drivers

• Executes at memory speed

Stub Driver

Virtual Network

Page 33: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.

Usage Scenarios

4 Examples on Desktops and Servers

Page 34: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.34

Deploy

ProductionVM

ProductionVM

ProductionVM

ProductionVM

Develop-mentVM

Scenario #1: Testing and Deployment

QAVM

TestDevelop

Page 35: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.35

Testing and Deployment

Test and deployin VMware worlds

Testing & deployment was error-prone and expensive

Challenge Solution

“VMware allows us to deliver well- tested and more reliable solutions in a shorter time frame at substantially lower costs."

Major Wall StreetInvestment Banking Firm

Page 36: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.36

Scenario # 2:Server Consolidation

Web Server

App Server

Web Server

App Server

Database Server

Database Server

App Server

App Server

Web Server

Web Server

VMware MultipleWorlds + Physical Hardware

Page 37: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.37

Server Consolidation

Run each database in a VMware world

One database per oil well,one server per database

The Challenge The Solution

oil well photo

“We’re able to run up to 10 database servers on a single server, which allows us to provide mainframe levels of reliability and data security at much lower cost."

Page 38: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.38

Scenario #3: Application Compatibility

• Some applications require their OS

• Some solutions require multiple applications

• Appliances provide solutions

VMware in Appliances

Page 39: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.39

Intel Appliance

Linux

Cisco Content Engine 590

Windows 2000

RealPlayerServer

Media Server

IP chain

Page 40: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.40

Scenario #4:Security Solutions

• Traditional tension : Security vs. Usability• Secure systems are not that usable

•E.g: require some particular OS setups• Flexible systems are not that secure

•Many documented examples

• Virtual Machines allow:• Secure Host

•that ensures the security of the whole system• Flexible, Usable Virtual Machines

•that play no role in the security of the whole system

Page 41: Usenix Invited Talk

© 2001 VMware, Inc. All rights reserved.41

National Security Agency NetTop

ClassifiedVM

VPN

Internet VM

Firewall

SE-Linux