Cloud Computing with Red Hat Solutions

Sivaram ShunmugamRed Hat Asia Pacific Pte Ltd

sivaram@redhat.com

3

Linux Automation Details Red Hat's Linux Automation strategy for next-generation IT infrastructure

includes:● Red Hat Enterprise Linux● Integrated Virtualization technologies● Virtualization Management technologies● Red Hat Enterprise MRG

4

The Concept of the CLOUD

Web 2.0● Amazon EC2● Google App Engine

Utility Computing Software-as-a-Service (SaaS) Hardware-as-a-Service (HaaS) So where is Red Hat ?

CPU, RAM, DISKonline

Red Hat Enterprise Linux

MySQL Database

JBoss Middleware

Partner Applications

5

Red Hat Enterprise Linux 5 Latest generation of the world's leading commercial open source environment

● Comprehensive OEM and ISV certifications● Leadership quality, performance and security● Supports Intel & AMD 32-bit/64-bit, Itanium2, IBM Power and mainframe● Includes virtualization for x86, x86-64 and Itanium2 architectures● Includes Clustering and Cluster File Systems

6

Based on Xen Hypervisor (RHEL) Embedded hypervisor based on KVM Hypervisor (oVirt) Almost native performance

(para-virtualization) Integrated into the operating system

Fully Supported � end to end No Patch-lag

No separate certifications forhardware and software

Uses existing Linux devices drivers Dynamically allocate resources

CPUs, memory, disks, networking

Ability to Migrate guests between hosts(Live Migration)

High Availability for Virtual Machines Shared Data for Virtual Machines / Physical Hosts

Integrated Virtualization

7

Performance and Scalability

Red Hat provides the most scalable and high performance virtualization platform

Para-virtualization (optimized kernel) provides best virtualization performance● RHEL guests typically run 20% to 100% faster on RHEL than on traditional

virtualization solutions● Some workloads can see up to a 400% performance improvement● Any workload can be virtualized even highly IO intensive – eg, databases

Full Virtualization● RHEL provides enhanced drivers for fully virtualized guests● Includes Linux and Windows virtual machines

● “Para-virtualized drivers”● Significant performance increases with fully virtualized guests

8

Benefits of Red Hat Virtualization

RHEL provides the most scalable and flexible Virtualization platform

● Allows organizations to virtualize all work loads● Removes barriers to virtualizing high performance enterprise applications

● Eliminates the economic constraints on virtualization● Virtualize any workload without added cost

● Reduces complexity● Deploy a single platform that covers all your virtualization need

● Includes support for Windows XP, 2000 and 2003● Use consistent tools to deploy, manage and deploy your entire

infrastructure

9

Provides a standard management interface● Hypervisor agnostic

● Will work with multiple hypervisors● Stable API

● Shield users from hypervisor changes● Consistent tools across hypervisors

● eg. Same interface for Xen, QEMU, KVM, OpenVZ, etc● Scriptable

● Provides APIs for developers / tool vendors● Allows vendors to build cross platform tools

libvirt

10

Virtualization Management Tools

11

Red Hat Network

A systems management platform designed to provide complete lifecycle management of the operating system and applications.

A single solution for lifecycle management of compute resources● Installing and provisioning new systems

● Supports Physical and Virtual systems● Updating/patching operating system● Install and patch custom software● Manage configuration files● Monitoring performance● Redeploying for a new purpose

12

oVirt project delivers complete virtualization solution

● Hypervisor● Built on Linux kernel ● Managed using standards based tools

● Based on libvirt, CIM and LDAP● From Red Hat, the community or ISVs

● Virtualization Management Platform● Cross platform – multiple hypervisors● Built on open standards

● LDAP, Kerberos, Libvirt, etc● Scalable

● From 1 node to tens of thousands of nodes● From small host cluster to cloud computing infrastructure

● Secure● Integrated policy and audit framework

13

Virtualization Alone is Not Enough: Challenges and Requirements for Today's Large-Scale Computing

Most enterprises are running more and more distributed applications and workloads and dealing with an increasing amount of distributed data

Many enterprises have increasing demands for computing power while simultaneously needing to cut down on physical resource consumption (e.g. power, space)

Many enterprises have variable needs for their amounts of computing power● Virtualization and consolidation can only get you so far if you have

significant variability in computing requirements or server loads● Some enterprises can never get enough computing capacity

14

Promises and Challenges of Cloud Computing

● Cloud computing promises to deliver many of the benefits of virtualization via an external provider

● Flexibility, scalability, on-demand utilization, efficiency, etc.● Many companies are looking at a variety of cloud-based offerings to address their

IT Infrastructure challenges● Research: Folding@Home, BOINC, Fedora Nightlife● Enterprise: Amazon EC2, Google App Engine

● There are important issues to consider when evaluating cloud infrastructure, including:

● Do I need to integrate the services with anything local in my datacenter or organization?● Do I have any particular requirements about data security?● How large are my datasets?● Are there any issues with platform lock-in?● Do I have any requirements or SLA's for how quickly my results return?● Do I want to add capacity to my IT infrastructure or outsource my IT infrastructure?

15

About Red Hat Enterprise MRG● Integrated platform for high

performance distributed computing

● High speed, interoperable, open standard Messaging

● Deterministic, low-latency Realtime kernel

● High performance & throughput computing Grid scheduler for distributed workloads,

16

About MRG Grid● Brings advantages of scale-out and flexible deployment to any application● Delivers better asset utilization, allowing applications to take advantage of all

available computing resources● Dynamically provisions additional peak capacity for “Christmas Rush”-like

situations● Executes across multiple platforms and in virtual machines● Schedules from sub-second jobs to long-running batch jobs● Provides seamless and flexible High Throughput Computing (HTC) and High

Performance Computing (HPC) across● Local grids● Remote grids● Remote clouds (Amazon EC2)● Cycle-harvesting from desktop PCs

17

MRG Grid is Based on Condor● MRG Grid is based on the Condor Project created and hosted by the University of

Wisconsin, Madison● Red Hat and the University of Wisconsin have signed a strategic partnership

around Condor:● University of Wisconsin makes Condor source code available under OSI-approved open

source license● Red Hat & University of Wisconsin jointly fund and staff Condor development on-

campus at the University of Wisconsin

● Red Hat and the University of Wisconsin's partnership will:● Add enhanced enterprise features, management, and supportability to Condor and MRG

Grid● Add High Throughput Computing capabilities to Linux

18

Red Hat is Initially Adding to Condor:● Enterprise Supportability

● Break out Condor from statically-linked blob to multiple well-maintained and individually patchable rpm's

● Web-Based Management Console● Unified management across all of MRG for job, system, and workload

management/monitoring

● AMQP Messaging Integration● Enable job submission to Condor via AMQP Messaging clients● Enable sub-second, low-latency scheduling for sub-second jobs

● Virtualization Support via libvirt Integration● Support scheduling of virtual machines on Linux using libvirt API's

● Amazon EC2 Integration● Enable automatic EC2 provisioning, job submission, results storage, teardown via

Condor scheduler● Runs as a job, so it can be a dependency for other jobs or executed based on rules

(e.g. add capacity in EC2 if local grid out of capacity)

19

Automating Virtualization with Red Hat Enterprise MRG MRG's Grid Scheduler can automatically and fully manage virtual machines via

libvirt: start, stop, migrate, etc

Integrating virtualization with MRG enables enterprises to:● Dynamically partition their data center based on workload priorities and load● Drive higher utilization by leveraging Condor's Compute On Demand to

schedule lower priority jobs onto idle servers● Drive higher utilization by optimizing resource matching via virtualization.

e.g. Schedule a Windows job onto a Linux resource● Schedule virtual machines onto idle desktop workstations and live migrate

them to other machines when the workstations are no longer idle

Red Hat Enterprise MRG builds on Red Hat's integrated virtualization to provide the tools to build your own internal cloud

and treat your IT infrastructure as a generic computational grid

20

MRG and Amazon EC2: The Most Flexibility and Best of all Worlds for Grid Deployments● Transparently move between or blend local grids with cloud-based grids

● Red Hat Enterprise Linux application certifications hold whether local or in Amazon EC2● Red Hat Enterprise MRG provides a unified management interface across local grids,

remote grids, and the cloud

● Dynamically add capacity in EC2 to dedicated grids where there is a need to maintain local infrastructure but there is a desire for flexible capacity

● e.g. Financial Services, Government, Healthcare

● Schedule entirely in EC2 through MRG when you only have occasional needs for grids. Easily add local capacity to your MRG Grid when necessary

● e.g. Internet, Manufacturing, Research

● No lock-in to local or cloud infrastructure

21

MRG and Amazon EC2 Demo Video

22

Enter SCS Grid ... Red Hat is one of the integral platform members as part of the

SCS-HP Grid Consortium

As part of the consortium, we are bringing infrastructure solutions to the Grid● Operating System platform● Red Hat Network for systems management, provisioning and monitoring● Applications through Partner ISV solutions

● Customer Relationship Management● Retail Management System● SMS Gateways● More on the SAAS track tomorrow.

23

Enterprise Cloud Architecture

Orc

hes

trat

ion

OperatingSystem

Hardware

VM OS

VM OS

VM OS

VM OS

VM OS

VM OS

VM OS

Gri

d S

ched

ule

r

Em

bed

ded

or

Vir

tH

yper

viso

r

Management

24

Enterprise Cloud Architecture

Orc

hes

trat

ion

OperatingSystem

Hardware

VM OS

VM OS

VM OS

VM OS

VM OS

VM OS

VM OS

Gri

d S

ched

ule

r

Em

bed

ded

or

Vir

tH

yper

viso

r

Management

RHEL

RHEL

RHEL

RHEL

RHEL

RHEL

RHEL

Co

nd

or

RHN

RHEL

AM

QP

+ o

Vir

t

RH

EL

/

oV

irt

RHELJBossMySQL

Postgres

25

Summary Red Hat is leveraging Linux Automation to deliver next-generation IT infrastructure

● Includes Integrated Virtualization that provides a scalable and high performance virtualization platform

● Provides virtualization management tools and open API's with libvirt● Provides grid scheduling capabilities for virtualization and cloud computing● Run any application, anywhere, anytime

Red Hat Enterprise Linux Virtualization: http://www.redhat.com/rhel/virtualization/

Red Hat Enterprise MRG: http://www.redhat.com/promo/mrg/