XPDS13: Xen Hypervisor Development Update - George Dunlap, Citrix

Intro OSS Development Xen 4.3 Xen 4.4 Updates

Xen Development update

George Dunlap

Edinburg – 21-23 October, 2013

Overview

I Principles of OSS Development

I Xen development process

I Review of 4.3 development process

I 4.4 development: Plans and predictionsI Project updates

I EFII Network driversI FreeBSDI libvirt integrationI NUMA

Edinburg – 21-23 October, 2013 Xen Development update 2 / 24

Overview

I 4.4 development: Plans and predictions

I Project updates

Overview

I Network driversI FreeBSDI libvirt integrationI NUMA

Overview

I EFII Network drivers

I FreeBSDI libvirt integrationI NUMA

Overview

I EFII Network driversI FreeBSD

I libvirt integrationI NUMA

Overview

I EFII Network driversI FreeBSDI libvirt integration

I NUMA

Overview

Principles

I Open-source: A force multiplier

I Limiting factor: Trust

I Goal: Meritocracy

Principles

I Goal: Meritocracy

Principles

I Goal: Meritocracy

Practice: Code development

I All patches posted to list

I Discussion and consensusI Committers are “referees”

I Determine that consensus has happenedI Break ties when consensus is not achieved

I Similar process for bigger decisions

I Discussion and consensus

I Committers are “referees”

I Determine that consensus has happened

I Break ties when consensus is not achieved

Releases

What we want:

I Visibility into the work that’s going on

I Clear picture of the state everything is in

I A roadmap of important featuresI A good release

I A bug-free releaseI An awesome releaseI An on-time release

Releases

What we want:

Releases

What we want:

I A roadmap of important features

I A good release

Releases

What we want:

Releases

What we want:

I A bug-free release

I An awesome releaseI An on-time release

Releases

What we want:

I A bug-free releaseI An awesome release

I An on-time release

Releases

What we want:

Releases

I Keep a list of outstanding features and bugs

I Things actively being worked onI Things we think would be good to work on

I Send the list every few weeks with updatesI Mirror on wiki:

I wiki.xen.org/wiki/Xen Roadmap/4.4

Releases

I Keep a list of outstanding features and bugsI Things actively being worked on

I Things we think would be good to work on

Releases

I Keep a list of outstanding features and bugsI Things actively being worked onI Things we think would be good to work on

Releases

I Send the list every few weeks with updates

I Mirror on wiki:

Releases

Release phases

I Open development

I Feature freeze: No new features (3 weeks)

I Code “freezing point”: Begin rejecting features (3 weeks)

I RCs: Bug fixes only (6 weeks)

Release phases

I Open development

Release phases

I Open development

Release phases

I Open development

Interlude

Xen 4.3 Planning Retrospective

I Goal: 9 month release cycle

I Scheduled release: June 2013

I Actual release: July 2013

I No “howlers”

Aug ’12

Jan ’13 Jul ’13

NUMA-aware scheduler (NUMA affinity) 3

ARM server port 3

openvswitch integration 3

16 TiB 3

backend scripts 3

qemu stubdoms 3

event channel limit 3

NUMA memory migration 3

virtual NUMA topology 3

PVH mode 3

pv audio 3

blktap3 3

vm import/export 3

xl USB hot-plug for HVM

libvirt libxl driver

XSM can replace IS PRIV

vTPM updates

osstest

qxl spice

Aug ’12 Jan ’13

Jul ’13

NUMA-aware scheduler (NUMA affinity) 3 3

ARM server port 3 3

openvswitch integration 3 7

16 TiB 3 3

backend scripts 3 3

qemu stubdoms 3 3

event channel limit 3 3

NUMA memory migration 3 3

virtual NUMA topology 3 3

PVH mode 3 3

pv audio 3 7

blktap3 3 7

vm import/export 3 7

xl USB hot-plug for HVM

libvirt libxl driver

XSM can replace IS PRIV

vTPM updates

osstest

qxl spice

Aug ’12 Jan ’13

Jul ’13

NUMA-aware scheduler (NUMA affinity) 3 3

ARM server port 3 3

openvswitch integration 3 7

16 TiB 3 3

backend scripts 3 3

qemu stubdoms 3 3

event channel limit 3 3

NUMA memory migration 3 3

virtual NUMA topology 3 3

PVH mode 3 3

pv audio 3 7

blktap3 3 7

vm import/export 3 7

xl USB hot-plug for HVM 3

libvirt libxl driver 3

XSM can replace IS PRIV 3

vTPM updates 3

osstest 3

qxl spice 3

Aug ’12 Jan ’13 Jul ’13NUMA-aware scheduler (NUMA affinity) 3 3 3

ARM server port 3 3 3

openvswitch integration 3 7 3

16 TiB 3 3 3

backend scripts 3 3 3

qemu stubdoms 3 3 7

event channel limit 3 3 7

NUMA memory migration 3 3 7

virtual NUMA topology 3 3 7

PVH mode 3 3 7

pv audio 3 7 7

blktap3 3 7 7

vm import/export 3 7 7

xl USB hot-plug for HVM 3 7

libvirt libxl driver 3 3

XSM can replace IS PRIV 3 3

vTPM updates 3 3

osstest 3 3

qxl spice 3 7

Don’t try to predict.

Xen 4.4 Planning

I Feature freeze: 18 October (past)

I Code freeze: Extended to catch a few straglers

Xen 4.4 Planning

Things that look likely:

I Multi-vector MSI (done)

I Improved spice support for libxl (done)

I PVH for domUs

I Non-udev scripts for driver domains(NetBSD driver domains)

I A ton of fixes from Coverity reports

I NUMA per-vcpu affinity

I USB hotplug for libxl

Xen 4.4 Planning

I PVH for domUs

Xen 4.4 Planning

I PVH for domUs

Xen 4.4 Planning

I PVH for domUs

Xen 4.4 Planning

I PVH for domUs

Xen 4.4 Planning

I PVH for domUs

Xen 4.4 Planning

I PVH for domUs

EFI intro

I Replacement for BIOS

I Includes “bootloader”-like functionality

I Boot-time services (similar to BIOS calls)

I Secure-boot services

EFI intro

EFI: Xen

(from Daniel Kiper @ Oracle)

I EFI support was introduced in Xen 4.2

I Only EFI loader is fully supported now

I GRUB/GRUB2 and Xen does not work on EFI platform

I But GRUB2 with fakebios and chainloader can be used to loadxen.efi on EFI platform (this option was not tested deeply)

I Daniel Kiper from Oracle works on full GRUB2 support

I Support for Secure Boot with GRUB2 will not be available atfirst

I GRUB2 requires some changes to support Secure Boot

EFI: Xen

EFI: Dom0 kernels

I Full Dom0 support is in SUSE Linux Kernel

I Upstream Linux Kernel does not have EFI support in Dom0yet

I Liang Tang from Oracle started working on it but laterdropped it

I There are some updates appearing from time to time onXen-devel but there is no WIP per se yet

EFI: Dom0 kernels

EFI: Guest support

I Open Virtual Machine Firmware (OVMF) was added to Xensource but it does not compile

I Framebuffer issue

EFI: Guest support

I Open Virtual Machine Firmware (OVMF) was added to Xensource but it does not compile

I Framebuffer issue

Network updates

(from Wei Liu @ Citrix)I Two new features during last cycle:

I 1:1 model netbackI split event channels

I Feature being worked on:

I IPv6 packet offload

I Planned new features:

I multiqueue supportI multipage ring support

Network updates

I 1:1 model netback

I split event channels

Network updates

I Feature being worked on:I IPv6 packet offload

Network updates

I Planned new features:I multiqueue support

I multipage ring support

Network updates

I Planned new features:I multiqueue supportI multipage ring support

FreeBSD Xen status

(from Roger Pau Monne @ Citrix)Xen FreeBSD updates present in 10.0 (to be released by the end ofNovember):

I Rework of event channel implementation so it is sharedbetween PV and PVHVM guests.

I Implementation of the vector callback for HVM (allowsper-cpu event channel interrupts).

I PV event timer and time counter implementation for PVHVM.I PV IPIs for PVHVM.I Live Migration support for PVHVM guests.I Support for FLUSH/BARRIER in blkfront.I Full Xen support merged into GENERIC kernel (no need to

compile a Xen specific kernel for PVHVM).I Bug fixes for OSS Xen, XenServer and Amazon EC2.

This work has been done in conjunction between Spectra Logicand Citrix.

FreeBSD Xen status

I PV event timer and time counter implementation for PVHVM.

I PV IPIs for PVHVM.I Live Migration support for PVHVM guests.I Support for FLUSH/BARRIER in blkfront.I Full Xen support merged into GENERIC kernel (no need to

FreeBSD Xen status

I PV event timer and time counter implementation for PVHVM.I PV IPIs for PVHVM.

I Live Migration support for PVHVM guests.I Support for FLUSH/BARRIER in blkfront.I Full Xen support merged into GENERIC kernel (no need to

FreeBSD Xen status

I PV event timer and time counter implementation for PVHVM.I PV IPIs for PVHVM.I Live Migration support for PVHVM guests.

I Support for FLUSH/BARRIER in blkfront.I Full Xen support merged into GENERIC kernel (no need to

FreeBSD Xen status

I PV event timer and time counter implementation for PVHVM.I PV IPIs for PVHVM.I Live Migration support for PVHVM guests.I Support for FLUSH/BARRIER in blkfront.

I Full Xen support merged into GENERIC kernel (no need tocompile a Xen specific kernel for PVHVM).

I Bug fixes for OSS Xen, XenServer and Amazon EC2.

FreeBSD Xen status

compile a Xen specific kernel for PVHVM).

I Bug fixes for OSS Xen, XenServer and Amazon EC2.

FreeBSD Xen status

(from Roger Pau Monne @ Citrix)What’s coming next:

I PVH guest support.

I PVH Dom0 support.

FreeBSD Xen status

(from Roger Pau Monne @ Citrix)What’s coming next:

I PVH guest support.

I PVH Dom0 support.

Block improvements

(from Roger Pau Monne @ Citrix)

I Persistent grants.

I Indirect descriptors.

Block improvements

(from Roger Pau Monne @ Citrix)

I Persistent grants.

I Indirect descriptors.

Libvirt

(from Dario Faggioli @ Citrix)

I Basic VM workflow

I Create, start, shutdown, reboot, destroy, pause, resume, save,restore, ... : working

I Migration, PCI passthrough: coming soon

I More advanced VM manipulation

I Scheduling, updating memory/vcpus, block attach, mediachange: working

I Vcpu pinning, NUMA node-affinity, network attach, native (xl)config file handling: missing

I Specific Libvirt stuff (for host and guests)

I Host and guest configuration and statistics reporting: missingI Block/network performance tuning and control: missing

Libvirt

I Basic VM workflowI Create, start, shutdown, reboot, destroy, pause, resume, save,

restore, ... : working

I Migration, PCI passthrough: coming soon

Libvirt

restore, ... : workingI Migration, PCI passthrough: coming soon

Libvirt

I More advanced VM manipulationI Scheduling, updating memory/vcpus, block attach, media

change: working

Libvirt

change: workingI Vcpu pinning, NUMA node-affinity, network attach, native (xl)

config file handling: missing

Libvirt

I Specific Libvirt stuff (for host and guests)I Host and guest configuration and statistics reporting: missing

I Block/network performance tuning and control: missing

Libvirt

I Specific Libvirt stuff (for host and guests)I Host and guest configuration and statistics reporting: missingI Block/network performance tuning and control: missing

I NUMA aware automatic placementWhen a VM is created, allocate its memory from theminimum number and best possible node(s)Included in Xen 4.2

I NUMA aware scheduling for Credit1VMs can run everywhere, but prefer to run on the nodeswhere their memory isIncluded in Xen 4.3

I Per-VCPU NUMA aware schedulingEach VCPU in a VM can have a different NUMAnode-affinity: different VCPUs (may) prefer to run ondifferent nodes Targeting Xen 4.4 (status: probable)

NUMA, con’t

I Virtual NUMA topology for PV guestsVMs can be NUMA machines and have a virtual topology(number or nodes, amount of memory, distances, etc.)Targeting Xen 4.4 (status: probable)

I Memory migrationMoving the memory of a VM from a NUMA node to anotheron-lineTargeting Xen 4.4 (status: tentative)

NUMA, con’t

I Virtual NUMA topology for PV guestsVMs can be NUMA machines and have a virtual topology(number or nodes, amount of memory, distances, etc.)Targeting Xen 4.4 (status: probable)

I Memory migrationMoving the memory of a VM from a NUMA node to anotheron-lineTargeting Xen 4.4 (status: tentative)

XPDS13: Xen Hypervisor Development Update - George Dunlap, Citrix

Technology

Transcript of XPDS13: Xen Hypervisor Development Update - George Dunlap, Citrix