Status Report

Ian PrattUniversity of Cambridge and Founder of XenSource Inc.

Computer Laboratory

Overview

Xen Today: 2.0.5 Xen 3.0 Development Update New benchmark results Ongoing research

Xen Today : 2.0 Features

Secure isolation between VMs Resource control and QoS Only guest kernel needs to be ported

All user-level apps and libraries run unmodified Linux 2.4/2.6, NetBSD, FreeBSD, Plan9

Execution performance is close to native Supports the same hardware as Linux x86 Live Relocation of VMs between Xen nodes

Para-Virtualization in Xen

Arch xen_x86 : like x86, but replace privileged instructions with Xen hypercalls Avoids binary rewriting and fault trapping For Linux 2.6, only arch-dep files modified

Modify OS to understand virtualised env. Wall-clock time vs. virtual processor time

• Xen provides both types of alarm timer Expose real resource availability

• Enables OS to optimise behaviour

MMU virtualisation: direct vs. shadow mode

I/O Architecture

Xen IO-Spaces delegate guest OSes protected access to specified h/w devices Virtual PCI configuration space Virtual interrupts

Devices are virtualised and exported to other VMs via Device Channels Safe asynchronous shared memory transport ‘Backend’ drivers export to ‘frontend’ drivers Net: use normal bridging, routing, iptables Block: export any blk dev e.g. sda4,loop0,vg3

Xen 2.0 Architecture

Event Channel Virtual MMUVirtual CPU Control IF

Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)

NativeDeviceDriver

GuestOS(XenLinux)

Device Manager & Control s/w

VM0

NativeDeviceDriver

GuestOS(XenLinux)

UnmodifiedUser

Software

VM1

Front-EndDevice Drivers

GuestOS(XenLinux)

UnmodifiedUser

Software

VM2

Front-EndDevice Drivers

GuestOS(XenBSD)

UnmodifiedUser

Software

VM3

Safe HW IF

Xen Virtual Machine Monitor

Back-End Back-End

Xen 3.0 Architecture

Event Channel Virtual MMUVirtual CPU Control IF

Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)

NativeDeviceDriver

GuestOS(XenLinux)

Device Manager & Control s/w

VM0

NativeDeviceDriver

GuestOS(XenLinux)

UnmodifiedUser

Software

VM1

Front-EndDevice Drivers

GuestOS(XenLinux)

UnmodifiedUser

Software

VM2

Front-EndDevice Drivers

UnmodifiedGuestOS(WinXP))

UnmodifiedUser

Software

VM3

Safe HW IF

Xen Virtual Machine Monitor

Back-End Back-End

VT-x

32/64bit

AGPACPIPCI

SMP

3.0 Headline Features

AGP/DRM in dom0 ACPI/PCI support in dom0 Support for SMP guests x86_64 support Intel VT-x support for unmodified guests Enhanced control and management

tools Optimised inter-VM networking IA64 and Power support, PAE36

x86_64

AMD Opteron and Intel EM64T Requires different approach to plain x86

Can’t use segmentation to protect Xen from guest OS kernels

Switch page tables between kernel and user Large VA space offers other optimisations

Current design supports up to 8TB mem Call for user testing in ~2-3 weeks

SMP Guest OSes

Takes great care to get good performance while remaining secure

Paravirtualized approach yields many benefits Avoids many virtual IPMIs

Need for better SMP-aware scheduler

Believed stable, optimisations pending

VT-x / Pacifica

Enables unmodified GuestOSes to be supported

Xen has excellent Shadow page table support

Requires simple platform emulation Install paravirtualized drivers after

booting for high-performance IO

4th Generation Tools

Controlling Xen is easy, it’s coordinating the rest of the system that’s hard Driver domains; firewall/routeing rules;

shaping LVM / filesystem image management VM relocation Resource measurement, control

Managing clusters of Xen nodes Replace monolithic xend with tool suite

communicating via The Registry

Physical Host

Dom0 DomU

xenbus netfront blkfrontnetback blkback xenbus

consoledaemon

ctrldaemon

xenbusdaemon

libxc / libxen

mux

registry

cluster managementtool

other clustermanagementtool

Live VM Relocation

Why is VM relocation useful? Managing a pool of VMs running on a cluster Taking nodes down for maintenance Load balancing VMs across the cluster

Why is it a challenge? VMs have lots of state Some VMs will have soft real-time

requirements• E.g. web servers, databases, game servers

Can only commit limited resources to migration

VM Relocation Strategy

Writeable Working Set

Rate Limited Migration

Iterative Progress: SPECWeb

Iterative Progress: Quake3

Quake 3 Server migration

Research Roadmap

Cluster load balancing Pre-migration analysis phase Optimization over coarse timescales

Evacuating nodes for maintenance Move easy to migrate VMs first

Storage-system support for VM clusters Decentralized, data replication, copy-on-

write “Internet Suspend Resume”

Just rsync plus IPSec tunnels

Research Roadmap

Cluster load balancing algorithms Exploit properties of live migration

System debugging and fault tolerance Lightweight checkpointing, distributed

watchpoints, deterministic replay I/O interposition and replay

VM forking Lightweight service replication, isolation

Secure virtualization Multi-level secure Xen

Conclusions

Xen 3.0 release on-target!