SUSE® Linux Enterprise Server as a Virtualization HostXen and KVM Hypervisors

Bruce RogersSoftware Engineer

brogers@suse.com

Michal SvecProduct Manager

msvec@suse.com

2

Agenda

Background

Supported Features and Guests

Using Xen and KVM: Tooling

Choosing Xen or KVM

Future: SUSE Linux Enterprise 12

3

What's Virtualization?

• What is it?‒ Hides the physical characteristics of a computing platform from

users, instead showing another abstract computing platform

• Containment and Virtualization‒ Hard containment = Virtualization

‒ Soft containment = LXC Containers – supported in SLES 11

‒ Fits well with some usage models, allowing less overhead than Xen or KVM

‒ Session TT1455 discusses LXC support in SLES 11

Background

5

Virtualization – A Short History

• Anciently, IBM's s/360 was first production system supporting Virt. SUSE now includes KVM Technology Preview support on s/390 systems

• SLES Virtualization centers on x86 architecture

‒ Full Virtualization (FV) w/out hardware (HW) assist difficult, but VMware finally succeeded

‒ Paravirtualization (PV) and then HW assist made it more viable and commonplace

• Xen Origins

‒ Xen began before x86 HW assist leveraging PV methods

‒ Xen adopted FV support with HW assist

• KVM Origins

‒ KVM project started as a result of HW assist

6

Comparing Xen and KVM

• Hypervisor Types‒ Type 1: Native or Bare Metal (eg: Xen)

‒ Type 2: Hosted (eg: KVM)

• Paravirtualization vs Full Virtualization‒ PV usually more efficient, but ...

‒ Advanced HW Virtualization features can beat PV methods

• Non-x86 Virtualization‒ S390x/System z in Technical Preview

• SUSE Linux Enterprise Server 11 SP3 versions ‒ Linux kernel: v3.0, qemu-kvm: v1.4.2, Xen: v4.2.2

7

Xen Architecture

8

KVM Architecture

Virtualization in SUSE Linux Enterprise

10

SUSE Virtualization

• Xen

‒ SUSE first to deliver Xen to the Enterprise in SLES 10 GA

‒ SUSE will continue to support Xen in SLE 12

• KVM

‒ SUSE first to deliver KVM to the Enterprise in SLES 11 GA

‒ SUSE first to deliver KVM on IBM System z in SLES 11 SP3

Virtualization is a key component in SUSE strategy!

11

Key Virtualization Pillars

• Full Support for leading open source hypervisors included in the platform – KVM, Xen and LXC

• Perfect Guest strategy, operating system tuned to run great as a guest on all major hypervisors

• SLES in the Clouds – work with major CSPs like Amazon, Microsoft, Dell, HP

• SUSE Lifecycle Management Story – SUSE Studio, SUSE Manager and SUSE Cloud

• Unique Cost Efficiency – unlimited number of virtual machines per physical server with one subscription

12

Virtualization Host

• Full Support for leading open source hypervisors included in the platform – KVM and Xen

• Support for x86_64, IBM System z (Tech Prev)

• Support for major virtualization features, incl. memory, network or block device hotplugging, VM save/restore, CPU or memory overcommitment, live migration, snapshots

• Support for OS-level virtualization with Linux Containers (LXC), allowing for better resource utilization

13

Perfect Guest

• Perfect Guest strategy, operating system tuned to run as an optimized guest on all major hypervisors, including

‒ VMware ESX

‒ Microsoft Hyper-V

‒ Citrix XenServer

‒ Oracle VM

‒ LPAR and z/VM for IBM System z

‒ SUSE Linux Enterprise Server with KVM, Xen and LXC

‒ SUSE Cloud

‒ Public Clouds

SUSE Linux Enterprise Server 11 SP3

15

SLES 11 SP3 Release Highlights

• Major features‒ Increased CPU and memory limits

‒ Enablement for latest hardware features, incl. new CPUs

‒ Performance improvements

‒ Support for SMEP

• Improved guest support‒ Microsoft Hyper-V, VMware ESX, SLES with Xen and KVM

‒ Support for Windows Server 2012 and Windows 8

‒ Increased performance and improved reliability and stability of PV drivers

• Easy to use resource management‒ LXC, control groups

16

SLES 11 SP3 New Xen Features

• EFI boot, Secure boot support

• Per device interrupt remapping

• New credit scheduler tuning parameters

• Multiple PCI segment support

• 5 TB host memory support

• Libvirt xen driver improvements

• Continue to use xend toolstack• Tech Preview: nested virtualization, libxl

Xen 4.2

17

SLES 11 SP3 New KVM Features

• IBM System z as host and guest

• AHCI device, virtio-scsi, USB passthrough

• Most command line parameters and monitor commands

• qed and qcow2 image format live migration

• Increased guest vcpu and ram limit

• Trim and hole-punch

• seccomp2 sandboxing

• Non-root user support

• Tech Preview: nested virtualization, virtio-blk-data-plane

KVM 3.0, QEMU 1.4.2

18

SLES 11 SP3: virtio-blk-dataplane

IBM System x3850 X5 host server with QLogic QLE 256x host bus Adapters and SLES 11 SP3

19

Key Additional Features in KVM and Xen

• Guest life-cycle controls‒ Startup/shutdown/reboot, pause, suspend/resume,

live migration, disk snapshot and deltas

• Paravirtual block and net devices

• qcow2, vmdk and raw disk image formats

• host device passthrough

• Memory, vcpu, and storage overcommit

• Memory ballooning

20

Key QEMU Based Features(QEMU provides device model for Xen HVM and KVM)

• Emulation of a PC style SMP hardware platform

• I440fx chipset with Xen/KVM specific cpu details

• ISA, PCI (including hotplug) and USB buses

• IDE and floppy storage interfaces and devices

• Common network adapters, sound cards

• Various display, keyboard and mouse options

• System BIOS, PXE BIOS's, boot control

21

Key Additional Features in Xen Only

• Optimized PV kernel‒ More complete, robust, performant, and mature than pv-ops

• Pygrub and DomULoader

• PV and FV (HVM) guests‒ Specify (limit) cpu features for guest

• PV guests‒ Virtualization on old hardware without HW assist

‒ vcpu hotplug

‒ PvSCSI

22

Key Additional Features in KVM Only

• VirtFS: file system “passthrough”

• Transparent Huge Page (THP) optimized

• Kernel Samepage Merging (KSM) supported

• Vhost-net kernel module accelerated networking

• Linux Guest Agent

23

SLES 11 SP3 and Linux Containers

• Support for system containers

• A full SLES installation into a chroot directory structure

• Resource control using cgroups

• Bridged networking required

• Only SLES 11 SP3 supported in container

24

SLES 11 SP3 Supported Guests (1/2)

SLES 11 SP3

SLES 11 SP2

SLES 10 SP4

SLED 11 SP3 (Technology Preview)

OES 2 SP3

OES 11 SP1

NetWare PV 6.5 SP8 (32-bit)

RHEL 5 (best effort)

RHEL 6 (best effort)

25

SLES 11 SP3 Supported Guests (2/2)

MS Windows 2003 SP2+

MS Windows 2008 SP2+

MS Windows 2008 R2+

MS Windows 2012+

MS Windows XP SP3+ (best effort)

MS Windows Vista SP2+ (best effort)

MS Windows 7 SP1+ (best effort)

MS Windows 8+ (best effort)

26

SLES 11 SP3 Xen Resource Limits

Guest Limits:

Max VMs per Host 128

Max virtual CPUs per VM 64 FV, 128 PV

Max Memory per VM 512 GiB

Max virtual block devices per VM pvscsi

Max virtual network cards per VM 8

Host Limits:

Max physical CPUs 256

Max physical memory 5 TiB

Max domain 0 physical memory 500 GiB

Max block devices up to 12,000 SCSI logical units

Max iSCSI devices 128

Max network cards 8

Max VMs per CPU core 8

Max virtual network cards 64 across all VMs in the system

27

SLES 11 SP3 KVM Resource Limits

Guest Limits:

Max VMs per host ≤ 8 * host cores

Max virtual CPUs per VM 160

Max Memory per VM 2 TiB

Max virtual block devices per VM virtio-scsi

Max virtual network cards per VM 8

Host Limits (equal to limits for SLES):

Max physical CPUs 4096

Max physical memory 16 TiB / 64 TiB

Using Xen and KVM: Tooling

29

Using Xen and KVM

• Using libvirt and libvirt tools to access Xen and KVM is recommended

‒ Includes: vm-install, virt-manager, virt-viewer, virsh commands

‒ Adds additional security, configurability, compatibility, etc.

• Using qemu-kvm command-line also supported – documentation identifies supported parameters

• Using xm commands supported with Xen 4.2 – libxenlight (xl) commands are unsupported

• Qemu-img image management tool provided

• The SUSE® Virtual Machine Driver Pack (VMDP) provides Xen and KVM drivers for Windows guests

30

libvirt

• Virtualization library for managing one host‒ Domains, networks, storage, host devices, ...

• Share application stack between hypervisors‒ Xen, qemu/kvm, LXC, VMware, VirtualBox, ...

• Long-term API/ABI stability and compatibility

• Integration with other SUSE Linux Enterprise components

‒ AppArmor, SELinux, CGroups, Linux Audit Framework, PolicyKit, ...

31

libvirt Tools

• virsh

‒ In-tree command line application exposing libvirt API

• vm-install and virt-inst

‒ Create virtual hardware configuration

‒ Install an OS in a virtual machine

• virt-viewer

‒ Graphical console client for virtual machines

• virt-manager

‒ Graphical tool for administering virtual machines

• vhostmd

‒ Metrics communication channel between host and virtual machines

• libvirt-cim

‒ libvirt-based implementation of DMTF Virtualization Management standards

Choosing Xen or KVM

33

Choosing a Virtualization Solution1 of 2

• Consider soft vs hard containment based on requirements

• Xen and KVM considered mostly on par, differentiated primarily by hypervisor type (type 1 vs type 2)

• Some argue that Xen's design is inherently more secure. There is a lot more involved in security than just design. There doesn't appear to be any clear indicators that Xen is more secure, but the Xen community seems a bit more security conscious

• KVM's Linux integration contributes to its popularity

• There is no general performance advantage of one over the other

34

Choosing a Virtualization Solution2 of 2

• Does PV inherently perform better than FV? Only if there isn't more optimized support built into hardware

‒ Xen PV x86_64 guest is inherently a bit inefficient

‒ Only Xen supports HW without HW Assist

‒ Xen continues to improve performance by using the best from HW Assist and PV constructs.

• KVM has more SUSE supported features than Xen

• Xen has a good footprint in large cloud deployments

• QEMU has undergone lots of “careful” churn

• Xen: xm → xl churn may force some to re-evaluate choice of Xen vs KVM going forward

35

Xen KVM

Ease of use

Performance

Security

Maturity

Features

Comparing Xen and KVM Redux

YesA matterof opinion?

We hopeHelpful?

Virtualization FutureSUSE Linux Enterprise Server 12

37

The Future: SLES 12 Virtualization1 of 3

• Include latest upstream release Anticipated versions:

‒ Xen v4.4

‒ KVM in v3.12 kernel

‒ QEMU v1.8.x

‒ Libvirt v1.1.7

• Xen Features‒ Continued Xen Support - Upstream Xen still very active!

‒ Libxenlight / xl interface (xend is gone!), incl. upgrade path

‒ pvUSB

‒ Use recent upstream QEMU internally

38

The Future: SLES 12 Virtualization2 of 3

• KVM Features‒ Linux and Windows Guest-Agent

‒ Improved NUMA Support

‒ hotplug memory

‒ hotplug vcpus

‒ vfio support

‒ vTPM

‒ Multiqueue networking

‒ SPICE – limited, remote server UI

39

The Future: SLES 12 Virtualization3 of 3

• Virtualization Tools

‒ p2v tool

‒ v2v tool (Xen to KVM)

‒ libguestfs

‒ Extensive v2v documentation

‒ Switch to virt-inst as default installer

‒ SPICE support in tools – limited, remote server UI

40

The Future: SLES 12 VirtualizationLinux Containers (LXC)

• Application containers support‒ Just the application being started within the container

• Easy application containers creation and management

• Integration with libvirt-LXC

• SELinux and AppArmor support and LXC

• Filesystem copy-on-write (btrfs integration)

41

Virtualization Management

1. Virt Manager‒ Simple tool, supports 1:1, 1:many

‒ Included directly in SLES

2. Virtualization Management‒ Covered by option 3 plus more

‒ SUSE Manager enhancements

3. Cloud Infrastructure‒ Automation, self-service, workload automation

‒ SUSE Cloud

42

Virtualization in the Cloud Ecosystem

SUSE Manager- Provisioning- Management- Monitoring

SUSE StudioBuilding workloadsfor physical andcloud environments

SUSE Linux EnterpriseThe foundation for your datacenter workloadsand virtualization

SUSE CloudHighly flexible and adaptable cloudinfrastructure

Where Can I Find More Information?

44

Virtualization ResourcesSUSE Linux Enterprise

Product Info‒ http://www.suse.com/products/server/‒ http://www.suse.com/products/vmdriverpack/

Documentation‒ SLES 11 Virtualization with KVM Administration Guide

‒ SLES 11 Virtualization with Xen Administration Guide

‒ SLES 11 Virtualization with LXC Quickstart

‒ SLES 11 Xen to KVM Migration Guide

Virtualization White-Papers‒ SUSE Linux Enterprise Server: Virtualization Technology Support

‒ KVM Virtualized I/O Performance

Corporate HeadquartersMaxfeldstrasse 590409 NurembergGermany

+49 911 740 53 0 (Worldwide)www.suse.com

Join us on:www.opensuse.org

45

Unpublished Work of SUSE. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.

General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.