Denali National Park and Preserve Final South Denali Implementation Plan And Environmental
The Denali Project
description
Transcript of The Denali Project
The Denali Project
Andrew Whitaker, Marianne Shaw, Rick Cox, Steven D. Gribble
http://denali.cs.washington.edu{andrew,rick,mar,gribble}@cs.washington.edu
A Brief History of VMMs
• Introduced by IBM in the late 1960’s:– CP/67 (1966)
– VM/370 (1972)
– z/VM (2004)
• Found wide use as a time-sharing system system– Ideological competitor to Multics, UNIX
• Today: VMMs have enjoyed a popularity resurgence – Industry: VMWare, Virtual PC (Microsoft), IBM
– Academia: Disco, Denali, ReVirt, Xen
Virtualizable Architectures
• Previous systems focused on strict virtualizability [Goldberg, 74]– Goal: minimize OS
changes
– VMM strictly emulates the phys. hardware
virtual arch
phys arch
hardware
…VM
VMM=
VM VM VM
The Denali Philosophy
• Key insight: software is not hardware– We can modify the VMM to achieve other goals
• Two ways to exploit change:– Modifying the virtual architecture [OSDI 2002]
– Changing the VMM implementation [NSDI 2004]
Outline
• Virtual Machine Monitor Background
• Denali and Architectural Changes– Motivation and approach– Architecture changes– Results
• Denali and Implementation Changes
Motivation
• Growing importance of untrusted code:– Internet services: ASP model, service authors
upload code into hosting center– PlanetLab: “slice” distributed platforms amongst
competing experiments, apps– Your desktop: Java{script}, untrusted binaries,
email attachments
• Needed: strong isolation platform– must be secure, efficient, and extensible
Why we need a new system
• Previous platforms are are inadequate:– Java sandboxes, ActiveX controls, system call
monitoring, virtual private servers, enhanced reference monitors
• Why?– require complex security policy– complexity, wide OS API– performance and/or scaling bottlenecks
Basic approach
• Focus on Internet services– Little sharing required
• Straw man: one service per physical machine– not cost-effective for many
services
• Instead, use virtual machines to emulate separate hardware
hardware
VMM
Web
server
Game
server
Overlay
router
VMMs provide strong isolation
• namespace isolation– no sharing possible, except through network
– eliminates security policy problems
• type-I VMMs run on bare hardware– no layer-below vulnerabilities, unfettered resource
management ability
• no high-level abstractions– eliminates complexity, reduces width of API
But, VMMs do too much!
• VMMs support isolation and full HW emulation– though some VMMs are
moving away from this…
• Isolation kernels separate the goals of isolation and emulation– make a clean break from
the underlying phys arch.
Goal VMM Isolation kernel
strong isolation
full hardware emulation
simplicity, scale, performance
Outline
• Virtual Machine Monitor Background
• Denali and Architectural Changes– Motivation and approach– Architecture changes– Results
• Denali and Implementation Changes
The Denali Virtual Architecture
• Retain x86 instruction set for performance
• Change other aspects to gain simplicity, scale, performance– gives up legacy OS
support
virtual arch
phys arch
hardware
…VM
Denali
VM VM VM
Arch. Changes for Simplicity
• Real hardware is unnecessarily complex
• Denali “scrubs” the hardware architecture:– Omit rarely used features
– Revise other features (interrupts, MMU)
– Streamlined hardware devices• Hardware Ethernet drivers make 12 PIOs per send
• Denali’s Ethernet requires single PIO
Source Code Complexity
0
10000
20000
30000
40000
50000
60000
70000
Linux Denali
Source lines
core kernel
arch. dependent
drivers
0
10000
20000
30000
40000
50000
Linux Denali
Source lines
storage
networking
virtual memory
other
Denali’s core kernel is roughly an order-of-magnitude smaller than bare-bones Linux 2.4.16
a) total lines b) core kernel breakdown
Arch. Change for Scale• Denali exposes an idle-with-timeout instruction
• Avoids a 50% throughput reduction for many VMs
0
100
200
300
400
500
0 50 100 150 200 250 300# virtual machines
Served load (req/sec)
idle-with-timeout
idle-without-timeout
Arch. Change for Performance
• Denali uses asynchronous, batched interrupt delivery
– avoiding context switches results in a 30% performance improvement
VM 1 VM 2 VM 3 VM 4scheduled VM
virtual interrupts 4 4
physical interrupts 4 4 4 4
time
Outline
• Virtual Machine Monitor Background
• Denali and Architectural Changes– Motivation and approach– Architecture changes– Results
• Denali and Implementation Changes
Experimental Setup
• Hardware– Intel Pentium 4 CPU; 1 GB RAM– 3 IDE disks– Intel PRO/1000 gigabit Ethernet NIC
• Software: apps run on Ilwaco guest OS– includes threads package, TCP/IP stack, libc– doesn’t use Denali MMU
• apps linked against Ilwaco like a libOS
Ethernet Virtualization Overhead
• Virtualization overhead is less than 20%
user/kernel crossing 3% , 2%
VM’s device driver
1% , 1%
TCP Stack
37% , 42%
read( )
packet copy VNIC demux Ethernet driver
12% , 13% 5% , 4% 43% , 39%
Ethernetpacketarrival
Breakdown of UDP receive costs (100 byte , 1400 byte packets)
total cost (cycles): 43477, 49002
Web Server Performance
• BSD: one kernel crossing per syscall
• Denali: one kernel crossing per packet
0
100
200
300
400
500
600
1 10 100 1000
document size (KB)
throughput (Mb/s)
BSD Mb/sec
Denali Mb/sec
0
100
200
300
400
500
600
1 10 100 1000document size (KB)
throughput (Mb/s)
BSD Mb/sec
BSD-syscall Mb/sec
Denali Mb/sec
Swapping in Denali
• Mechanism: demand-loading of “virtual” physical memory from statically allocated swap
• Factors that influence performance:– # disks– memory size– popularity distribution– block size– data locality– striping policy
Scale in Denali: 10,000 Web Servers
0
1000
2000
3000
4000
5000
6000
0 200 400 600 800 1000 1200 1400
# VMs
req/sec
0
10
20
30
40
0 2000 4000 6000 8000 10000# VMs
req/sec
In memory performance Disk-bound performance
disk paging bandwidth
• Denali utilizes most of the available hardware resources
memoryexceeded
Outline
• Virtual Machine Monitor Background
• Denali and Architectural Changes– Motivation and approach– Architecture changes– Results
• Denali and Implementation Changes
The VMM as a Service Platform
• Adding services to a VMM:– Migration (Intel Research Pittsburgh, Stanford)
– Intrusion detection (Stanford)
– System replay (Michigan)
– Fault Tolerance (Cornell)
• Advantages:– Virtual devices are relatively simple
– VMM has a “whole-system” perspective
Denali: An Extensible VMM
• Problem: existing VMMs are hard to extend– New services require significant re-implementation or reverse-
engineering
Denali approach:– Factor out VMM design decisions (disk, swap, etc.)
• “Parent” virtual machines can implement functionality on behalf of their “children”
– Encapsulate these decisions with clean interfaces• Supply default implementations for commonly used components
A Denali Interface
// the virtual Disk device callback functionsstruct Disk {
int (*diskWrite)(char *buffer, int offset, int num_sectors);
int (*diskRead)(char *buffer, int offset, int num_sectors);
int (*getSizeInSectors)(void);
int (*getSectorSize)(void);
} Disk;
Migrating an Apache Web Server
0
50
100
150
200
250
300
0 5 10 15 20 25
Time (seconds)
Req/secMachine A
Machine B
Implemented in only 289 lines of C source code
Migration
Conclusions
• Denali exploits the degree-of-freedom offered by software to enhance VMMs– Architecture changes for scale,
performance, simplicity
– Implementation changes for providing virtual machine services