Post on 01-Dec-2014
description
FC Draft 8152008
RHEL and Xen on Itanium Open Source Virtualization with Enterprise Class
Features
2008 FOCUS Consulting wwwfocusonsystemscom Page 1
Red Hat Enterprise Linux Virtualization and Intel Itanium Linux Open Source Virtualization with Enterprise-Class Features
A FOCUS White Paper August 2008
FOCUS Consulting
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 1
Table of Contents
Executive Summary 2
State of the Industry 2
Drivers for Virtualization 2
Drivers for Enterprise-Class Features on Servers 3
Drivers for Open Source 4
Background and History of Xen 5
Solution Overview ndash Red Hat Enterprise Linux on Itanium 5
Xen and Red Hat Enterprise Linux 5
Key Features 6
Guest Operating Systems 6
Management 7
Licensing 8
Intel Itanium Overview 8
Features 9
Intel VT-i 10
Key Benefits and Challenges 11
Conclusions and Recommendations 11
FOCUS Assessment 12
References 12
Other Related FOCUS Reports 12
About FOCUS 13
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 2
Executive Summary
Server virtualization is a powerful enabling technology that offers a compelling business case for consolidating multiple server workloads onto one physical server using server virtualization software Leveraging this capability increases the criticality of the server hardware mdash if a server fails many applicationsworkloads are affected rather than just one As a result consolidating business-critical workloads drives a need for enterprise mainframe-like features into the server
In a virtual environment obtaining enterprise mainframe-class features requires tightly interconnected hardware firmware virtualization software and guest operating system solutions Red Hat Enterprise Linux Virtualization on Intel Itanium with VT-i extensions is a good example of a powerful combined solution The use of multiple technologies from the open source community including Linux Xen libvirt and others brings the work of the best and brightest together in this solution Additionally Red Hat leverages standards such as libvirt and common information model (CIM) to provide management based on open standards allowing organizations to select the tools that best fit their environment
This software combination coupled with Intelrsquos highly reliable available and serviceable Itanium processor with Machine Check Architecture (MCA) and Intel VT-i virtualization assist provides a foundation for mainframe-class computing without mainframe pricing and maintenance expenses
This white paper discusses the state of the industry detailing market drivers for mainframe-class features on a server and the benefits of open source technology solutions explaining the background and history of open source virtualization beginning with the Xen hypervisor It details Red Hat Enterprise Linux Virtualization for Itanium describing salient features and provides the architectural overview of both the hardware and software components of the solution Key benefits and challenges are covered as well as conclusions and recommendations
State of the Industry
The compelling business case around server consolidation using server virtualization has gained the attention of C-level executives around the world
Drivers for Virtualization
It is estimated that most servers today are utilizing less than 10 of available capacity Using server virtualization to consolidate these underutilized resources drives up utilization on the remaining systems This increases the return on the server investment and reduces the total cost of server ownership through reductions in space power cooling management and hardware maintenance
IDC estimates that 90 of all very large corporations (companies with more than 10000 employees) and between 70 and 80 of medium-to-large corporations (with 1000 to 10000 employees) have implemented server virtualization in one form or another However current
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 3
estimates indicate that only 10 of all servers are currently virtualized This means that there is still substantial growth yet to occur for server virtualization throughout corporate enterprises In addition to the early dominance of VMware and other niche offerings the market is continuing to mature with the entry of many new server virtualization solutions including Microsoft Hyper-V and a list of companies incorporating the Xen hypervisor including Red Hat Novell Citrix Virtual Iron Sun and Oracle
Server consolidation forces the requirement to make the system highly available This section describes the market drivers for mainframe-class servers and for open source server virtualization including a brief history of the Xen Open Source Hypervisor
Drivers for Enterprise-Class Features on Servers
The need for enterprise-class features in servers has been growing over the past 10 years starting with the explosive growth of the Internet Additionally the shift to a more global market has accelerated the need for cost-effective commodity servers that have to be up and running 24x7 The cost of downtime is high and constantly increasing More recently the popularity of consolidating servers using virtualization has raised the issue of ldquoputting all your eggs in one basketrdquo As a result the need for high reliability availability and serviceability of servers running virtual workloads has moved to the forefront
With the single application per server model used by most IT organizations in the past when a server failed only one application was at risk Today with many application workloads consolidated on a single server using server virtualization (see Figure 1) a failure at the server level can lead to disastrous results
Figure 1 Server virtualization consolidates physical servers and their application workloads onto one server
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 4
Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed
The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe
Drivers for Open Source
Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available
In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software
Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community
Definition of Terms
ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory
Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i
Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer
Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors
Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts
Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine
Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware
Virtual Machine Monitor ndash Another term for
hypervisor
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 5
Background and History of Xen
Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)
The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)
In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems
In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen
Solution Overview ndash Red Hat Enterprise Linux on Itanium
The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management
Xen and Red Hat Enterprise Linux
Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i
The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 6
access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers
Key Features
Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance
With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot
Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources
Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure
Guest Operating Systems
As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 7
Figure 2 High-level RHEL 51 for Itanium architecture
Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported
Management
There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release
The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments
Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 1
Table of Contents
Executive Summary 2
State of the Industry 2
Drivers for Virtualization 2
Drivers for Enterprise-Class Features on Servers 3
Drivers for Open Source 4
Background and History of Xen 5
Solution Overview ndash Red Hat Enterprise Linux on Itanium 5
Xen and Red Hat Enterprise Linux 5
Key Features 6
Guest Operating Systems 6
Management 7
Licensing 8
Intel Itanium Overview 8
Features 9
Intel VT-i 10
Key Benefits and Challenges 11
Conclusions and Recommendations 11
FOCUS Assessment 12
References 12
Other Related FOCUS Reports 12
About FOCUS 13
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 2
Executive Summary
Server virtualization is a powerful enabling technology that offers a compelling business case for consolidating multiple server workloads onto one physical server using server virtualization software Leveraging this capability increases the criticality of the server hardware mdash if a server fails many applicationsworkloads are affected rather than just one As a result consolidating business-critical workloads drives a need for enterprise mainframe-like features into the server
In a virtual environment obtaining enterprise mainframe-class features requires tightly interconnected hardware firmware virtualization software and guest operating system solutions Red Hat Enterprise Linux Virtualization on Intel Itanium with VT-i extensions is a good example of a powerful combined solution The use of multiple technologies from the open source community including Linux Xen libvirt and others brings the work of the best and brightest together in this solution Additionally Red Hat leverages standards such as libvirt and common information model (CIM) to provide management based on open standards allowing organizations to select the tools that best fit their environment
This software combination coupled with Intelrsquos highly reliable available and serviceable Itanium processor with Machine Check Architecture (MCA) and Intel VT-i virtualization assist provides a foundation for mainframe-class computing without mainframe pricing and maintenance expenses
This white paper discusses the state of the industry detailing market drivers for mainframe-class features on a server and the benefits of open source technology solutions explaining the background and history of open source virtualization beginning with the Xen hypervisor It details Red Hat Enterprise Linux Virtualization for Itanium describing salient features and provides the architectural overview of both the hardware and software components of the solution Key benefits and challenges are covered as well as conclusions and recommendations
State of the Industry
The compelling business case around server consolidation using server virtualization has gained the attention of C-level executives around the world
Drivers for Virtualization
It is estimated that most servers today are utilizing less than 10 of available capacity Using server virtualization to consolidate these underutilized resources drives up utilization on the remaining systems This increases the return on the server investment and reduces the total cost of server ownership through reductions in space power cooling management and hardware maintenance
IDC estimates that 90 of all very large corporations (companies with more than 10000 employees) and between 70 and 80 of medium-to-large corporations (with 1000 to 10000 employees) have implemented server virtualization in one form or another However current
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 3
estimates indicate that only 10 of all servers are currently virtualized This means that there is still substantial growth yet to occur for server virtualization throughout corporate enterprises In addition to the early dominance of VMware and other niche offerings the market is continuing to mature with the entry of many new server virtualization solutions including Microsoft Hyper-V and a list of companies incorporating the Xen hypervisor including Red Hat Novell Citrix Virtual Iron Sun and Oracle
Server consolidation forces the requirement to make the system highly available This section describes the market drivers for mainframe-class servers and for open source server virtualization including a brief history of the Xen Open Source Hypervisor
Drivers for Enterprise-Class Features on Servers
The need for enterprise-class features in servers has been growing over the past 10 years starting with the explosive growth of the Internet Additionally the shift to a more global market has accelerated the need for cost-effective commodity servers that have to be up and running 24x7 The cost of downtime is high and constantly increasing More recently the popularity of consolidating servers using virtualization has raised the issue of ldquoputting all your eggs in one basketrdquo As a result the need for high reliability availability and serviceability of servers running virtual workloads has moved to the forefront
With the single application per server model used by most IT organizations in the past when a server failed only one application was at risk Today with many application workloads consolidated on a single server using server virtualization (see Figure 1) a failure at the server level can lead to disastrous results
Figure 1 Server virtualization consolidates physical servers and their application workloads onto one server
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 4
Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed
The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe
Drivers for Open Source
Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available
In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software
Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community
Definition of Terms
ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory
Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i
Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer
Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors
Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts
Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine
Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware
Virtual Machine Monitor ndash Another term for
hypervisor
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 5
Background and History of Xen
Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)
The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)
In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems
In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen
Solution Overview ndash Red Hat Enterprise Linux on Itanium
The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management
Xen and Red Hat Enterprise Linux
Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i
The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 6
access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers
Key Features
Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance
With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot
Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources
Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure
Guest Operating Systems
As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 7
Figure 2 High-level RHEL 51 for Itanium architecture
Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported
Management
There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release
The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments
Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 2
Executive Summary
Server virtualization is a powerful enabling technology that offers a compelling business case for consolidating multiple server workloads onto one physical server using server virtualization software Leveraging this capability increases the criticality of the server hardware mdash if a server fails many applicationsworkloads are affected rather than just one As a result consolidating business-critical workloads drives a need for enterprise mainframe-like features into the server
In a virtual environment obtaining enterprise mainframe-class features requires tightly interconnected hardware firmware virtualization software and guest operating system solutions Red Hat Enterprise Linux Virtualization on Intel Itanium with VT-i extensions is a good example of a powerful combined solution The use of multiple technologies from the open source community including Linux Xen libvirt and others brings the work of the best and brightest together in this solution Additionally Red Hat leverages standards such as libvirt and common information model (CIM) to provide management based on open standards allowing organizations to select the tools that best fit their environment
This software combination coupled with Intelrsquos highly reliable available and serviceable Itanium processor with Machine Check Architecture (MCA) and Intel VT-i virtualization assist provides a foundation for mainframe-class computing without mainframe pricing and maintenance expenses
This white paper discusses the state of the industry detailing market drivers for mainframe-class features on a server and the benefits of open source technology solutions explaining the background and history of open source virtualization beginning with the Xen hypervisor It details Red Hat Enterprise Linux Virtualization for Itanium describing salient features and provides the architectural overview of both the hardware and software components of the solution Key benefits and challenges are covered as well as conclusions and recommendations
State of the Industry
The compelling business case around server consolidation using server virtualization has gained the attention of C-level executives around the world
Drivers for Virtualization
It is estimated that most servers today are utilizing less than 10 of available capacity Using server virtualization to consolidate these underutilized resources drives up utilization on the remaining systems This increases the return on the server investment and reduces the total cost of server ownership through reductions in space power cooling management and hardware maintenance
IDC estimates that 90 of all very large corporations (companies with more than 10000 employees) and between 70 and 80 of medium-to-large corporations (with 1000 to 10000 employees) have implemented server virtualization in one form or another However current
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 3
estimates indicate that only 10 of all servers are currently virtualized This means that there is still substantial growth yet to occur for server virtualization throughout corporate enterprises In addition to the early dominance of VMware and other niche offerings the market is continuing to mature with the entry of many new server virtualization solutions including Microsoft Hyper-V and a list of companies incorporating the Xen hypervisor including Red Hat Novell Citrix Virtual Iron Sun and Oracle
Server consolidation forces the requirement to make the system highly available This section describes the market drivers for mainframe-class servers and for open source server virtualization including a brief history of the Xen Open Source Hypervisor
Drivers for Enterprise-Class Features on Servers
The need for enterprise-class features in servers has been growing over the past 10 years starting with the explosive growth of the Internet Additionally the shift to a more global market has accelerated the need for cost-effective commodity servers that have to be up and running 24x7 The cost of downtime is high and constantly increasing More recently the popularity of consolidating servers using virtualization has raised the issue of ldquoputting all your eggs in one basketrdquo As a result the need for high reliability availability and serviceability of servers running virtual workloads has moved to the forefront
With the single application per server model used by most IT organizations in the past when a server failed only one application was at risk Today with many application workloads consolidated on a single server using server virtualization (see Figure 1) a failure at the server level can lead to disastrous results
Figure 1 Server virtualization consolidates physical servers and their application workloads onto one server
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 4
Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed
The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe
Drivers for Open Source
Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available
In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software
Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community
Definition of Terms
ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory
Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i
Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer
Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors
Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts
Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine
Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware
Virtual Machine Monitor ndash Another term for
hypervisor
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 5
Background and History of Xen
Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)
The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)
In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems
In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen
Solution Overview ndash Red Hat Enterprise Linux on Itanium
The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management
Xen and Red Hat Enterprise Linux
Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i
The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 6
access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers
Key Features
Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance
With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot
Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources
Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure
Guest Operating Systems
As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 7
Figure 2 High-level RHEL 51 for Itanium architecture
Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported
Management
There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release
The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments
Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 3
estimates indicate that only 10 of all servers are currently virtualized This means that there is still substantial growth yet to occur for server virtualization throughout corporate enterprises In addition to the early dominance of VMware and other niche offerings the market is continuing to mature with the entry of many new server virtualization solutions including Microsoft Hyper-V and a list of companies incorporating the Xen hypervisor including Red Hat Novell Citrix Virtual Iron Sun and Oracle
Server consolidation forces the requirement to make the system highly available This section describes the market drivers for mainframe-class servers and for open source server virtualization including a brief history of the Xen Open Source Hypervisor
Drivers for Enterprise-Class Features on Servers
The need for enterprise-class features in servers has been growing over the past 10 years starting with the explosive growth of the Internet Additionally the shift to a more global market has accelerated the need for cost-effective commodity servers that have to be up and running 24x7 The cost of downtime is high and constantly increasing More recently the popularity of consolidating servers using virtualization has raised the issue of ldquoputting all your eggs in one basketrdquo As a result the need for high reliability availability and serviceability of servers running virtual workloads has moved to the forefront
With the single application per server model used by most IT organizations in the past when a server failed only one application was at risk Today with many application workloads consolidated on a single server using server virtualization (see Figure 1) a failure at the server level can lead to disastrous results
Figure 1 Server virtualization consolidates physical servers and their application workloads onto one server
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 4
Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed
The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe
Drivers for Open Source
Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available
In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software
Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community
Definition of Terms
ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory
Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i
Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer
Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors
Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts
Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine
Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware
Virtual Machine Monitor ndash Another term for
hypervisor
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 5
Background and History of Xen
Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)
The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)
In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems
In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen
Solution Overview ndash Red Hat Enterprise Linux on Itanium
The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management
Xen and Red Hat Enterprise Linux
Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i
The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 6
access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers
Key Features
Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance
With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot
Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources
Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure
Guest Operating Systems
As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 7
Figure 2 High-level RHEL 51 for Itanium architecture
Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported
Management
There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release
The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments
Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 4
Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed
The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe
Drivers for Open Source
Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available
In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software
Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community
Definition of Terms
ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory
Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i
Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer
Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors
Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts
Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine
Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware
Virtual Machine Monitor ndash Another term for
hypervisor
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 5
Background and History of Xen
Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)
The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)
In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems
In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen
Solution Overview ndash Red Hat Enterprise Linux on Itanium
The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management
Xen and Red Hat Enterprise Linux
Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i
The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 6
access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers
Key Features
Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance
With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot
Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources
Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure
Guest Operating Systems
As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 7
Figure 2 High-level RHEL 51 for Itanium architecture
Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported
Management
There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release
The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments
Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 5
Background and History of Xen
Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)
The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)
In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems
In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen
Solution Overview ndash Red Hat Enterprise Linux on Itanium
The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management
Xen and Red Hat Enterprise Linux
Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i
The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 6
access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers
Key Features
Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance
With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot
Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources
Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure
Guest Operating Systems
As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 7
Figure 2 High-level RHEL 51 for Itanium architecture
Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported
Management
There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release
The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments
Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 6
access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers
Key Features
Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance
With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot
Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources
Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure
Guest Operating Systems
As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 7
Figure 2 High-level RHEL 51 for Itanium architecture
Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported
Management
There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release
The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments
Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 7
Figure 2 High-level RHEL 51 for Itanium architecture
Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported
Management
There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release
The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments
Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 8
information model provider for the Distributed Management Task Force (DMTF) virtualization schema
Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform
Red Hat delivers a number of management tools built on libvirt including
virsh ndash A command line shell enabling customers to script and automate virtualization management
Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management
Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring
Licensing
Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL
Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product
Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices
Intel Itanium Overview
The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 9
Features
All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture
A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously
RAS Feature Typical
Mainframe Intel Itanium
Platforms Intel Xeon
MP Platforms Intel Xeon Platforms
Cache ECC coverage
Memory single-device error correct
Memory retry on double-bit error
Error recovery on data bus (ECC)
Internal logic soft error checking
Badpoisoned data containment
Cache reliability
Memory sparing
Memory mirroring
Hot plug IO (PCI-X PCI Express)
Memory hot swap
Table 1 Dual-core Itanium processor error coverage
With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)
Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 10
Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs
The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware
Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture
Intel VT-i
The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 11
paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment
In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments
Key Benefits and Challenges
For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system
Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility
As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization
Conclusions and Recommendations
The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 12
FOCUS Assessment
Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices
References
ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf
ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance
Other Related FOCUS Reports
FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch
FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC
FOCUS Market Landscape Report Desktop and Application Delivery Alternatives
FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager
FOCUS Solution Profile Citrix XenServer
FOCUS Solution Profile VMware Desktop Solutions
All trademarks are the properties of their respective owners
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom
FC 8152008
Red Hat Enterprise Linux Virtualization and Intel Itanium
2008 FOCUS Consulting wwwfocusonsystemscom Page 13
About FOCUS
Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years
Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers
FOCUS delivers research analysis and consulting focused on systems software and storage
in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of
Technology (ROI TCO) wwwfocusonsystemscom