Open stack

http://slidesha.re/1gF0PEK1

Openstack => Cloud computing

at your fingertips!

Luan CestariFebruary 27 , 2014


Please, let me ask ...

● Have you heard about Cloud Computing?

● Is it a hype? What does it means?

● Are you using any cloud service?


Do you know OpenStack?

● Is the OpenStack an OS?

● Why would I use instead of the machine in my enterprise?

● What is this Open Source thing and what is related to this talking?


But what is OpenStack

● OpenStack is an open source project for

building a private or public

infrastructure-as-a-service (IaaS) cloud

● OpenStack solution != Virtualization -> let's see why● Cloud!

● But that doesn't explain what is the definition of cloud


The defining the Cloud Computing: It must be ...

● Scalable

● Portable

● On-demand

● Resource Management

● Measureable

From: National Institute of Standards and Technology (NIST)


8 out of 10 clouds are built on Linux1

Amazon EC2 RackSpace

1 Linux Adoption Trends 2012: A Survey of Enterprise End Users, Linux Foundation, January 18, 20122 Windows 8? It Won’t Win Microsoft’s Biggest Battle, Robert Mcmillan, Wired, October 25, 2012

“Linux is twice as popular as Windows on

Amazon Web Services.”2

“On the RackSpace cloud, the split is even starker: 75 %to 25 %,

again in favor of Linux.”2

Linux Powers The Cloud


ServerVirtualization

DistributedVirtualization

PrivateCloud

HybridCloud

Visibility

ControlOptimization

Automation

Agility

Self-Service

Federation

Brokering

Consolidation Reduce Capital Expense

Flexibility & Speed Reduce Operational Expense Automation Less Downtime

Self-Serve Agility Standardization IT as a Business Usage Metering

Choice of CAPEX/OPEX model Increased Flexibility (up and down)

Drivers

Virtual Infrastructure Management

Derived from Gartner Roadmap: From Virtualization to Cloud Computing (reference slide)

Virtualization to cloud infrastructure continuum


PrivateCloud

PublicClouds

HybridCloud

Service ProviderOwned and managed, Accessed via the web, Pay for what you use.

Privately owned And managed with Restricted access (but Could be externallyhosted)

Interoperable combination

of private and public cloud.

CommunityCloud

Cloud deployment models


Types of Cloud

● Saas (Software as a service)

● PaaS (Platform as a service)

● IaaS (Infrastructure as a service)


STORAGE(RHS)

HARDWARE(x86)

VIRTUALIZATION(RHEV)

OPERATING SYSTEM(RHEL)

APPLICATION PLATFORM(JBOSS, PHP, RUBY, ETC)

APPLICATION

Managed by the Public or Private Cloud Offering

Managed and Controlled by Customer (IT, Dev, or User)

IaaS PaaS SaaS

Increased Control

Reduced DIY

Cloud Service Models


13

A cloud provider view of shared responsibility for security

Source: Cloud Security Alliance


14


Software-as-a-Service (SaaS) (hosted apps)

Platform-as-a-Service (PaaS)(dev platform, apps middleware)

Infrastructure-as-a-Service (IaaS)(compute, storage, network)

End-users

DevelopersDevOps

IT Admins

Amazon AWS RackSpace OpenStack vCloud Director

OpenShift Force.com Azure

Google Apps Salesforce Many moreIaaS

PaaS

SaaS

PrivateClouds

PublicClouds

Hybrid

CommunityCloud

Cloud types & deployment models


16

With PaaSHow to Build an App:1. Have Idea2. Get Budget3. Code4. Test5. Launch6. Automatically Scale

How to Build an App:1. Have Idea2. Get Budget3. Submit VM Request request4. Wait5. Deploy

framework/appserver6. Deploy testing tools7. Test testing tools8. Code9. Configure Prod VMs10. Push to Prod11. Launch12. Request More Prod VMs to

meet demand13. Wait14. Deploy app to new VMs15. Etc.

VirtualizedHow to Build an App:1. Have Idea2. Get Budget3. Submit hardware acquisition

request4. Wait5. Get Hardware6. Rack and Stack Hardware7. Install Operating System8. Install Operating System

Patches/Fix-Packs9. Create user Accounts10. Deploy framework/appserver11. Deploy testing tools12. Test testing tools13. Code14. Configure Prod servers (and buy

them if needed)15. Push to Prod16. Launch17. Order more servers to meet

demand18. Wait…19. Deploy new servers20. Etc.

Physical

“The use of Platform-as-a-Service technologies will enable IT organizations to become more agile and more responsive to the business needs.” –Gartner*

Streamlining App Dev with PaaS

More info: www.openshift.com


Don’t take taxonomies too literally

● IaaS can blend into PaaS

● “Value-add” services like DynamoDB, Elastic MapReduce

● PaaS can blend into SaaS

● PaaS anchored to a SaaS environment

● Taxonomy part of broader ecosystem

● Hybrid cloud IaaS management (CloudForms)

● APIs/services

● Development tooling


Back to OpenStack, its history...

● 2010 - Rackspace and Nasa joins forces● Rackspace's Cloud Files platform and NASA's Nebula● OpenStack Object Store (Swift) and OpenStack

Compute Nova


Versions

Codename Date

K ?

Juno Oct 2014

Icehouse Apr 17, 2014

Havana Oct 17, 2013

Grizzly Apr 4, 2013

Folsom Sep 27, 2012

Essex Apr 5, 2012

Diablo Sep 22, 2011

Cactus Apr 15, 2011

Bexar Feb 3, 2011

Austin Oct 21, 2010


Overview simplified of OpenStack


Little zoom in


Little more


Components

● OpenStack Compute (Nova) - Austin● OpenStack Network Service (Quantum/Neutron) - Folsom● OpenStack Object Storage (Swift) - Austin● OpenStack Block Storage (Cinder) - Folsom● OpenStack Identity (Keystone) - Essex● OpenStack Image (Glance) - Bexar● OpenStack Dashboard (Horizon) - Essex● OpenStack Orchestration (Heat) - Havana● OpenStack Metering (Ceilometer) – Havana


Incubated Components

● OpenStack Database Service (Trove) ● OpenStack Bare Metal (Ironic)● OpenStack Queue Service (Marconi) ● OpenStack Data Processing (Savannah) ● TripleO/Tuskar● Oslo● TaskSystem-as-a-Service (Convection)● DNSaaS (Designate)● Application catalog (Murano)


More projects

● StackForger● DevStack● Tempest● Beaker● ...


Supported Hypervisors

● KVM● LXC (through libvirt)● QEMU● UML● VMWare vSphere● Xen● Hyper-V ● Bare Metal● Docker

More: https://wiki.openstack.org/wiki/HypervisorSupportMatrix


Supported Images

● raw● vhd● vmdk● iso● qcow2● vdi● aki● ari● ami


How does it look like


How to install OpenStack, the easy way

sudo yum install -y http://rdo.fedorapeople.org/rdo-release.rpm

sudo yum install -y openstack-packstack

packstack --allinone

http://rdo.fedorapeople.org/rdo-release.rpm

http://rdo.fedorapeople.org/rdo-release.rpm


RED HAT LEADS THROUGH OPEN INNOVATION

RED HAT ENTERPRISE LINUX – FOUNDATION FOR THE OPEN HYBRID CLOUDDOC144908-20130711R4

● Enterprise-hardened OpenStack software

● Delivered with an enterprise life cycle

● Six-month release cadence offset from community releases to allow testing

● Aimed at long-term production deployments

● Certified hardware and software through the Red Hat OpenStack Cloud Infrastructure Partner Network

● Supported by Red Hat

● Latest OpenStack software, packaged in a managed open source community

● Facilitated by Red Hat

● Aimed at architects and developers who want to create, test, collaborate

● Freely available, not for sale

● Six-month release cadence mirroring community

● No certification, no support

● Installs on Red Hat and derivatives

● Open source, community-developed (upstream) software

● Founded by Rackspace Hosting and NASA

● Managed by the OpenStack Foundation

● Vibrant group of developers collaborating on open source cloud infrastructure

● Software distributed under the Apache 2.0 license

● No certifications, no support

OpenStack Progression


● Installer appropriate for smaller scale OpenStack deployments.

● Driven by asking questions or an “answer file”

● Uses SSH and Puppet to set up all nodes

PackStack Overview


Upstream

DevStack

Unit Tests

SmokeStack

Tempest

Source from GIT

rdopkg

Torpedo / Tempest

RDO Environments

RDO Release / Poodle

Midstream

SmokeStack Trunk

Downstream

RHOS Environments

Packages from Brew

Tempest

Job Builder/Runner

RHOS Release

Differences between versions?


● Upstream:

● Contribute tempest tests upstream● Detect rpm based install errors via smokestack● Enterprise Linux devstack

● Midstream:

● Detect packstack, foreman based install errors● Qualify RDO across supported environments● Improve the feedback to development

● Downstream:

● Qualify RHOS across supported environments● Scale and Performance test



RHEL OPENSTACK PLATFORM VALUE

● Enterprise-grade OpenStack version with ecosystem, lifecycle, & support customers expect from Red Hat!

●

● Based on RHEL and includes required fixes in both OpenStack and RHEL

● Enterprise hardened OpenStack code

● Longer supported lifecycle (starts with 1 year for Folsom/Grizzly)

● Bug fixes, security errata, selected backports, etc.

● Certified ecosystem (Red Hat Certified OpenStack Partner Program and Red Hat Enterprise Linux ecosystem)

● Full support for RHEL and Windows workloads

Why Red Hat vs other community versions?


Red Hat Enterprise Linux Defines the Open Hybrid Cloud

OPEN

open innovation,

open standards,

open APIs,

openness vs. lockin

HYBRID

hybrid deployment models (physical, virtual, cloud)

hybrid architectures

public-private-hybrid cloud scenarios

CLOUD

Scalable

Portable

On-demand

Resource Management

Measureable


Red Hat Product Portfolio


More info in the notes of the slides

https://smokestack.openstack.org/


Questions?


Extra: more about OpenStack Architecture and how to scale

From Russell Bryant (Red Hat Summit)


OpenStack Identity (Keystone)

Compute

Dashboard

ImageService

Identity

ObjectStorage

MeteringOrchestration

NetworkingBlockStorage



● Important Concepts● Tenants – Groups of Users● Users● Roles● Tokens● Services



keystone

Token ServicesToken Identity

ReST API


keystone...

Load Balancer

keystone keystone

OpenStack Identity (Keystone) Scaling


OpenStack Dashboard (Horizon)

Compute

Dashboard

ImageService

Identity

ObjectStorage




OpenStack APIs


● Python WSGI Application

● Django framework

● Stateless client of OpenStack APIs

httpd

horizon


OpenStack APIs

OpenStack Dashboard (Horizon) Scaling

httpd

horizon

httpd

horizon

httpd

horizon...

Load Balancer

Session Storage


OpenStack Orchestration (Heat)

Compute

Dashboard

ImageService

Identity

ObjectStorage





● Consumes all other OpenStack APIs

● Important concept: template defined stacks


OpenStack APIs


heat-api

ReST API / CFN API

AMQP

heat-engine

DB


OpenStack APIs

OpenStack Orchestration (Heat) Scaling

heat-api

AMQP

heat-engine

heat-apiheat-api ...

Load Balancer

heat-engineheat-engine ...


OpenStack Metering (Ceilometer)

Compute

Dashboard

ImageService

Identity

ObjectStorage





● The foundation of billing or charge back systems

● Concepts● Meters● Compute Pollsters● Central Pollster● Notifications● Collectors



Credit: Doug Hellmanhttp://stevedore.readthedocs.org/en/latest/essays/pycon2013.html#requirements-for-ceilometer


OpenStack Object Storage (Swift)

Compute

Dashboard

ImageService

Identity

ObjectStorage





● Concepts● Accounts● Containers – Organize Your Data● Objects – Your Data● Ring – Internal Data Structure



Credit: Mark McLoughlin

API

Storage


Proxy Proxy Proxy...

Load balancer

Storage Storage Storage...

OpenStack Object Storage (Swift) Scaling


OpenStack Image Service (Glance)

Compute

Dashboard

ImageService

Identity

ObjectStorage





● Concepts● Images● Metadata● Storage Backends



glance-api

glance-registry

DBImage Storage

ReST API

ReST


OpenStack Image Service (Glance) Scaling

glance-api

glance-registry

DBImage Storage

glance-apiglance-api ...

Load Balancer

* Scales horizontally the same way as the API


OpenStack Block Storage (Cinder)

Compute

Dashboard

ImageService

Identity

ObjectStorage





● Concepts● Volumes● Snapshots● Storage Backends



cinder-api

ReST API

AMQP

cinder-volume

cinder-scheduler

DB


cinder-schedulercinder-scheduler

OpenStack Block Storage (Cinder) Scaling

cinder-api

AMQP

cinder-volume

cinder-scheduler

cinder-volumecinder-volume

cinder-api cinder-api...

Load Balancer


OpenStack Networking (...)

Compute

Dashboard

ImageService

Identity

ObjectStorage





● Concepts● Networks● Routers● Subnets● Ports● Vendor plugins


OpenStack Compute (Nova)

Compute

Dashboard

ImageService

Identity

ObjectStorage





● Concepts● Instances / Servers● Flavors / Instance Types● Virt drivers

● OpenStack API and EC2 API



nova-api

AMQP

nova-compute

Libvirt+KVM

DB

nova-scheduler

nova-conductor

ReST API


OpenStack Compute (Nova) Scaling (Step 1)

AMQP

Libvirt+KVM

DB

nova-scheduler

nova-conductor

Libvirt+KVMLibvirt+KVM

nova-compute

nova-api

Load Balancer



API Cell

Compute Cell Compute Cell Compute Cell...

AMQP



AMQP

Libvirt+KVM

DB

nova-scheduler

nova-conductor


nova-compute

Compute Cellnova-cells



AMQP

DB

API Cell

nova-cells

nova-api

Load Balancer

ReST API



API Cell


AMQP


OpenStack Components

Compute

Dashboard

ImageService

Identity

ObjectStorage




Boot a Server – Step 1

Compute

Dashboard

ImageService

Identity

ObjectStorage



1



Compute

Dashboard

ImageService

Identity

ObjectStorage



1 2



nova-api

AMQP

nova-compute

Libvirt+KVM

DB

nova-scheduler

nova-conductor

ReST API



Compute

Dashboard

ImageService

Identity

ObjectStorage



1 2

3



Compute

Dashboard

ImageService

Identity

ObjectStorage



1 2

3

4



Compute

Dashboard

ImageService

Identity

ObjectStorage



1 2

3

4

5



Compute

Dashboard

ImageService

Identity

ObjectStorage



1 2

3

4

5

6


Openstack => Cloud computing

at your fingertips!

Luan CestariFebruary 27 , 2014


Please, let me ask ...

● Have you heard about Cloud Computing?

● Is it a hype? What does it means?

● Are you using any cloud service?

ScalablePortableOn-demandResource ManagementMeasureable

3


Do you know OpenStack?

● Is the OpenStack an OS?

● Why would I use instead of the machine in my enterprise?

● What is this Open Source thing and what is related to this talking?


But what is OpenStack

● OpenStack is an open source project for

building a private or public

infrastructure-as-a-service (IaaS) cloud

● OpenStack solution != Virtualization -> let's see why● Cloud!

● But that doesn't explain what is the definition of cloud

The difference in http://www.slideshare.net/CAinc/cloud-expo-session-from-virtualization-to-cloud-computing-building-an-effective-pragmatic-reliable-cloud

5


The defining the Cloud Computing: It must be ...

● Scalable

● Portable

● On-demand

● Resource Management

● Measureable

From: National Institute of Standards and Technology (NIST)


Open standards: Advanced Research Projects Agency Network (ARPANET) collaborative process(in 60s) led to the birth of the Internet (in 1969)GNU project, 1983, Richard Stallman

The prior slide described the CIO dilemma being required to do more with less. This will not happen the old fashion way and a new more disruptive approach is needed. 1. more use of OSS2. new service delivery models – cloud3. new business models – pay as you go and subscription based

Most clouds are build on OSS and 80% of the workloads run on Linux. Open Standards are essential to ensure portability which is in the DNA of OSS

77


8 out of 10 clouds are built on Linux1

Amazon EC2 RackSpace

1 Linux Adoption Trends 2012: A Survey of Enterprise End Users, Linux Foundation, January 18, 20122 Windows 8? It Won’t Win Microsoft’s Biggest Battle, Robert Mcmillan, Wired, October 25, 2012

“Linux is twice as popular as Windows on

Amazon Web Services.”2

“On the RackSpace cloud, the split is even starker: 75 %to 25 %,

again in favor of Linux.”2

Linux Powers The Cloud

Today, Linux is the primary platform for a majority of cloud-based applications. As a case in point, an October 2012 Wired magazine article cited a report by Newvem which illustrated Linux's dominant position, particularly pertaining to Amazon Web Services:

At Wired’s request, Newvem — a company that sells management services to Amazon cloud customers — took at look at about 41,000 cloud machines run by several hundred customers. Its conclusion: Linux is twice as popular as Windows on Amazon Web Services. It was running on 67 percent of machines, compared to Windows’ 33 percent.

With Linux as the go-to OS for many cloud users, we are also seeing workloads migrate to Linux for public, private, and hybrid clouds because of efficiency and flexibility of deployment. Companies at the cutting edge of cloud computing and the Internet are choosing Linux and open source, and often choose Red Hat Enterprise Linux.

http://www.wired.com/wiredenterprise/2012/10/epic-microsoft-windows-fight/http://www.redhat.com/resourcelibrary/case-

studies/http://csrc.nist.gov/publications/nistpubs/800-

145/SP800-145.pdfhttp://www.ibm.com/developerworks/podcast/dwi/

feature040611-dnielsen.html

7


ServerVirtualization

DistributedVirtualization

PrivateCloud

HybridCloud

Visibility

ControlOptimization

Automation

Agility

Self-Service

Federation

Brokering

Consolidation Reduce Capital Expense

Flexibility & Speed Reduce Operational Expense Automation Less Downtime

Self-Serve Agility Standardization IT as a Business Usage Metering

Choice of CAPEX/OPEX model Increased Flexibility (up and down)

Drivers

Virtual Infrastructure Management

Derived from Gartner Roadmap: From Virtualization to Cloud Computing (reference slide)

Virtualization to cloud infrastructure continuum

This slide is derived from the Gartner roadmap “From Virtualization to Cloud Computing”, and indicates the various levels of infrastructure enablement that IT organizations are engaging in today. Organizations often field more than one virtual / cloud project may be doing IT transformation to IaaS while at the same time running Cloud program initiatives.

Focus has been on deriving economies and flexibility at one level via server virtualization and consolidation. Highly distributed and sprawled virtual environments that tend to result form those initiatives require management optimization and automation to be efffectivley controlled.

Infrastructure as a Service (IaaS) capabilities are then engaged to support agile, self-service access to IT systems and services, again raising the management ante for addressing tracking, capacity management, resource utilization/optimization,

In Hybrid Cloud, portable workloads across mixed hypervisors and policies to govern controlled access and usage are required, as well as chargeback and showback mechanisms.

With CLOUDForms, the goal is to future proof your management investment and eliminate multiple disparate tools and tool sprawl which introduces problems of integration, multiple interfaces, and rising costs & training needs with multiple vendor point products.

Allows your organization to incrementally build into cloud architectures at its own pace.


IT must change and be able to address the questions of the prior slide. If IT is not able to adapt and change the business as their customer will go around IT and buy services directly from public cloud and SaaS providers which cannot be in the interest of IT and the corporation overall because shadow IT will sprawlIT needs to provide the benefits of a public could service by eliminating the potential negative sides of public service like security concerns, governance, regulatory restrictions, ...


PrivateCloud

PublicClouds

HybridCloud

Service ProviderOwned and managed, Accessed via the web, Pay for what you use.

Privately owned And managed with Restricted access (but Could be externallyhosted)

Interoperable combination

of private and public cloud.

CommunityCloud

Cloud deployment models


Types of Cloud

● Saas (Software as a service)

● PaaS (Platform as a service)

● IaaS (Infrastructure as a service)


STORAGE(RHS)

HARDWARE(x86)

VIRTUALIZATION(RHEV)

OPERATING SYSTEM(RHEL)

APPLICATION PLATFORM(JBOSS, PHP, RUBY, ETC)

APPLICATION

Managed by the Public or Private Cloud Offering

Managed and Controlled by Customer (IT, Dev, or User)

IaaS PaaS SaaS

Increased Control

Reduced DIY

Cloud Service Models

To set the proper context for our discussion, let’s take a look at the three best known cloud service delivery models,

IaaS or Infrastructure as a Service,

PaaS or Platform as a Service,

And SaaS or Software as a Service.

With each of these some portion of the technology stack is managed and controlled by the Cloud offering (whether public or private), and the rest is responsibility of the customer of the Cloud offering. They must manage and maintain the portion of the stack that is not managed by the Cloud offering. The benefit is that the customer can also CONTROL the design of this portion of the stack.

● For Infrastructure-as-a-Service, or IaaS, the customer must manage and control everything from the Operating System up. They must install the OS, the middleware, and the application code. This ultimately becomes only a small reduction in effort compared to bare-metal server management.

● For Software-as-a-Service, or SaaS, the customer doesn’t need to manage anything, but they also don’t control anything, not even the design of the application. The complete application is delivered from the Cloud for the user to use. This can be great UNLESS there is some application functionality that the customer needs that is not available in the application. I.e., the business application offered via SaaS may not meet the actual business needs of the user.

● Platform-as-a-Service, or PaaS (commonly pronounced “Paz”) offers a happy medium and allows the customer to control the design of the application but not worry about the underlying components of the stack. The management and maintenance of the complete application platform stack is abstracted away. The customer, typically a Developer in this case, can focus on their application design, code, and ultimately their application users


13

A cloud provider view of shared responsibility for security

Source: Cloud Security Alliance


14


Software-as-a-Service (SaaS) (hosted apps)

Platform-as-a-Service (PaaS)(dev platform, apps middleware)

Infrastructure-as-a-Service (IaaS)(compute, storage, network)

End-users

DevelopersDevOps

IT Admins

Amazon AWS RackSpace OpenStack vCloud Director

OpenShift Force.com Azure

Google Apps Salesforce Many moreIaaS

PaaS

SaaS

PrivateClouds

PublicClouds

Hybrid

CommunityCloud

Cloud types & deployment models


16

With PaaSHow to Build an App:1. Have Idea2. Get Budget3. Code4. Test5. Launch6. Automatically Scale

How to Build an App:1. Have Idea2. Get Budget3. Submit VM Request request4. Wait5. Deploy

framework/appserver6. Deploy testing tools7. Test testing tools8. Code9. Configure Prod VMs10. Push to Prod11. Launch12. Request More Prod VMs to

meet demand13. Wait14. Deploy app to new VMs15. Etc.

VirtualizedHow to Build an App:1. Have Idea2. Get Budget3. Submit hardware acquisition

request4. Wait5. Get Hardware6. Rack and Stack Hardware7. Install Operating System8. Install Operating System

Patches/Fix-Packs9. Create user Accounts10. Deploy framework/appserver11. Deploy testing tools12. Test testing tools13. Code14. Configure Prod servers (and buy

them if needed)15. Push to Prod16. Launch17. Order more servers to meet

demand18. Wait…19. Deploy new servers20. Etc.

Physical

“The use of Platform-as-a-Service technologies will enable IT organizations to become more agile and more responsive to the business needs.” –Gartner*

Streamlining App Dev with PaaS

More info: www.openshift.com

PaaS allows us to make this leap from a Craftwork methodology to an Assembly Line methodology. PaaS both streamlines and standardizes the Application Development process.

Instead of the unique, one-off processes that each application project used to go through, Utilization of a PaaS allows the deployment of a standardized methodology with a few distinct repeatable steps.

Once the idea and budget are secured, then the App Dev team just simply starts working with the PaaS to code, test, and launch their application. PaaS provides the platform environment that allows Developers to start coding immediately. Server deployment, configuration and administration is all largely automated. This reduces the burden on Operations and reduces the delays that impact Development.

And many industry analysts agree that the utilization of PaaS is going to be the key to allowing IT to be the demands of the business.

<next slide>


Don’t take taxonomies too literally

● IaaS can blend into PaaS

● “Value-add” services like DynamoDB, Elastic MapReduce

● PaaS can blend into SaaS

● PaaS anchored to a SaaS environment

● Taxonomy part of broader ecosystem

● Hybrid cloud IaaS management (CloudForms)

● APIs/services

● Development tooling


Back to OpenStack, its history...

● 2010 - Rackspace and Nasa joins forces● Rackspace's Cloud Files platform and NASA's Nebula● OpenStack Object Store (Swift) and OpenStack

Compute Nova


Versions

Codename Date

K ?

Juno Oct 2014

Icehouse Apr 17, 2014

Havana Oct 17, 2013

Grizzly Apr 4, 2013

Folsom Sep 27, 2012

Essex Apr 5, 2012

Diablo Sep 22, 2011

Cactus Apr 15, 2011

Bexar Feb 3, 2011

Austin Oct 21, 2010


Overview simplified of OpenStack


Little zoom in


Little more


Components

● OpenStack Compute (Nova) - Austin● OpenStack Network Service (Quantum/Neutron) - Folsom● OpenStack Object Storage (Swift) - Austin● OpenStack Block Storage (Cinder) - Folsom● OpenStack Identity (Keystone) - Essex● OpenStack Image (Glance) - Bexar● OpenStack Dashboard (Horizon) - Essex● OpenStack Orchestration (Heat) - Havana● OpenStack Metering (Ceilometer) – Havana


Incubated Components

● OpenStack Database Service (Trove) ● OpenStack Bare Metal (Ironic)● OpenStack Queue Service (Marconi) ● OpenStack Data Processing (Savannah) ● TripleO/Tuskar● Oslo● TaskSystem-as-a-Service (Convection)● DNSaaS (Designate)● Application catalog (Murano)


More projects

● StackForger● DevStack● Tempest● Beaker● ...


Supported Hypervisors

● KVM● LXC (through libvirt)● QEMU● UML● VMWare vSphere● Xen● Hyper-V ● Bare Metal● Docker

More: https://wiki.openstack.org/wiki/HypervisorSupportMatrix

KVM - Kernel-based Virtual Machine. The virtual disk formats that it supports is inherited from QEMU since it uses a modified QEMU program to launch the virtual machine. The supported formats include raw images, the qcow2, and VMware formats.

LXC - Linux Containers (through libvirt), use to run Linux-based virtual machines.

QEMU - Quick EMUlator, generally only used for development purposes.

UML - User Mode Linux, generally only used for development purposes.

VMWare vSphere 4.1 update 1 and newer, runs VMWare-based Linux and Windows images through a connection with a vCenter server or directly with an ESXi host.

Xen - XenServer, Xen Cloud Platform (XCP), use to run Linux or Windows virtual machines. You must install the nova-compute service in a para-virtualized VM.

Hyper-V - Server virtualization with Microsoft's Hyper-V, use to run Windows, Linux, and FreeBSD virtual machines. Runs nova-compute natively on the Windows virtualization platform.

Bare Metal - Not a hypervisor in the traditional sense, this driver provisions physical hardware through pluggable sub-drivers (for example, PXE for image deployment, and IPMI for power management).

Docker is an open-source engine which automates the deployment of >applications as highly portable, self-sufficient containers which are >independent of hardware, language, framework, packaging system and hosting >provider.


Supported Images

● raw● vhd● vmdk● iso● qcow2● vdi● aki● ari● ami

KVM - Kernel-based Virtual Machine. The virtual disk formats that it supports is inherited from QEMU since it uses a modified QEMU program to launch the virtual machine. The supported formats include raw images, the qcow2, and VMware formats.

LXC - Linux Containers (through libvirt), use to run Linux-based virtual machines.

QEMU - Quick EMUlator, generally only used for development purposes.

UML - User Mode Linux, generally only used for development purposes.

VMWare vSphere 4.1 update 1 and newer, runs VMWare-based Linux and Windows images through a connection with a vCenter server or directly with an ESXi host.

Xen - XenServer, Xen Cloud Platform (XCP), use to run Linux or Windows virtual machines. You must install the nova-compute service in a para-virtualized VM.

Hyper-V - Server virtualization with Microsoft's Hyper-V, use to run Windows, Linux, and FreeBSD virtual machines. Runs nova-compute natively on the Windows virtualization platform.

Bare Metal - Not a hypervisor in the traditional sense, this driver provisions physical hardware through pluggable sub-drivers (for example, PXE for image deployment, and IPMI for power management).

Docker is an open-source engine which automates the deployment of >applications as highly portable, self-sufficient containers which are >independent of hardware, language, framework, packaging system and hosting >provider.



Log in, using your account from keystone



Create an instance

Give it a nameSet parameters (CPUs, RAM, Disk, ...)Shows you how your request stacks up against your available quota



Manage existing instances

Networking ParametersVNC ConsoleManage VM lifecycle (such as reboot and Terminate)


How to install OpenStack, the easy way

sudo yum install -y http://rdo.fedorapeople.org/rdo-release.rpm

sudo yum install -y openstack-packstack

packstack --allinone


RED HAT LEADS THROUGH OPEN INNOVATION

Gerry

RED HAT ENTERPRISE LINUX – FOUNDATION FOR THE OPEN HYBRID CLOUDDOC144908-20130711R4

● Enterprise-hardened OpenStack software

● Delivered with an enterprise life cycle

● Six-month release cadence offset from community releases to allow testing

● Aimed at long-term production deployments

● Certified hardware and software through the Red Hat OpenStack Cloud Infrastructure Partner Network

● Supported by Red Hat

● Latest OpenStack software, packaged in a managed open source community

● Facilitated by Red Hat

● Aimed at architects and developers who want to create, test, collaborate

● Freely available, not for sale

● Six-month release cadence mirroring community

● No certification, no support

● Installs on Red Hat and derivatives

● Open source, community-developed (upstream) software

● Founded by Rackspace Hosting and NASA

● Managed by the OpenStack Foundation

● Vibrant group of developers collaborating on open source cloud infrastructure

● Software distributed under the Apache 2.0 license

● No certifications, no support

OpenStack Progression

Chuck


● Installer appropriate for smaller scale OpenStack deployments.

● Driven by asking questions or an “answer file”

● Uses SSH and Puppet to set up all nodes

PackStack Overview

35


Upstream

DevStack

Unit Tests

SmokeStack

Tempest

Source from GIT

rdopkg

Torpedo / Tempest

RDO Environments

RDO Release / Poodle

Midstream

SmokeStack Trunk

Downstream

RHOS Environments

Packages from Brew

Tempest

Job Builder/Runner

RHOS Release


36


● Upstream:

● Contribute tempest tests upstream● Detect rpm based install errors via smokestack● Enterprise Linux devstack

● Midstream:

● Detect packstack, foreman based install errors● Qualify RDO across supported environments● Improve the feedback to development

● Downstream:

● Qualify RHOS across supported environments● Scale and Performance test


● In many ways, the RHOS and RHEL value propositions are identical with Red Hat OpenStack (RHOS) being to OpenStack what Red Hat Enterprise Linux (RHEL) is to Linux.

● Red Hat has the proven capability of preparing and delivering an open source distribution for enterprise use and providing a stable, secure, and common platform that customers, ISV partners, IHV partners, and service provider partners can use to test and certify their software and hardware solutions.

● Operational competency necessary for maintain a stable platform with backported bugfixes and features. World-class processes and procedures required for the rigorous testing and QA necessary for enterprise-class products

● Worldwide enterprise class support service with consulting and training services● Certified partner program● Guest certifications – include Microsoft Windows (SVVP)● Hardware certifications – broadest list of certified server, storage and networking

platforms● Leadership and influence of the upstream OpenStack product development process

with the ability and motivation to act as customer advocate with upstream community


Red Hat Enterprise Linux Defines the Open Hybrid Cloud

OPEN

open innovation,

open standards,

open APIs,

openness vs. lockin

HYBRID

hybrid deployment models (physical, virtual, cloud)

hybrid architectures

public-private-hybrid cloud scenarios

CLOUD

Scalable

Portable

On-demand

Resource Management

Measureable


Red Hat Product Portfolio


More info in the notes of the slides

Upstream CI http://www.youtube.com/watch?v=v0IWQP_m_38

● http://ci.openstack.org/devstack-gate.html● http://status.openstack.org/zuul● https://smokestack.openstack.org/

http://stackalytics.com/

Name:https://wiki.openstack.org/wiki/Release_Naming Stat:http://activity.openstack.org/dash/browser/http://stackalytics.com/http://www.ohloh.net/p/openstack

https://access.redhat.com/site/documentation/en-US/Red_Hat_OpenStack/2/html/Getting_Started_Guide/ch01.html

Ideas:http://openstack.redhat.com/OpenStack_Summit_Hong_Kong

Projects:About the project• Has separate developers and design teams• Has a well defined public API ( With the exception of Horizon and several other projects) -> have a RESTfull (JSON/HTTP) API• Has a separate database and isolated persistent layer Other projects:https://wiki.openstack.org/wiki/Marconi SQS (producer-consumer http://aws.amazon.com/sns/) and SNS (pub-sub http://aws.amazon.com/sqs/) from AWShttp://www.slideshare.net/kamesh001/open-stack-101https://wiki.openstack.org/wiki/OsloHA:many forms, there is many solutions; like AWS also uses HAProxy for HAhttp://openstack.redhat.com/Load_Balance_OpenStack_APIhttp://openstack.redhat.com/forum/discussion/268/rdo-with-fuel-is-that-possible/p1http://haproxy.1wt.eu/http://nginx.org/https://github.com/observing/balancerbattlehttps://github.com/eucalyptus/architecture/blob/master/features/elb/3.3/elb-benchmark.wikihttp://centminmod.com/siegebenchmark_nginx_test3.htmlhttp://centminmod.com/apachebench_nginx.htmlhttp://serverfault.com/questions/293131/advantages-of-using-nginx-or-ha-proxy-as-load-balancerhttp://stackoverflow.com/questions/13210636/differentiate-nginx-haproxy-varnish-and-uwsgi-gunicornhttp://codeblow.com/questions/proxy-options-mod-proxy-balancer-nginx-proxy-balancer-haproxy/http://37signals.com/svn/posts/1073-nuts-bolts-haproxy

Tools:packstack https://github.com/redhat-openstack/packstack http://blog.flaper87.org/post/511441160f06d34258e8a6ac/http://goodsquishy.com/2012/12/introducing-openstack-packstack/https://docs.google.com/presentation/d/1O9pf1J68ZSRV96mY2Zv8tzkvSqmoHbRWWbnVJcUOuM0/edit#slide=id.gbd379966_63foreman http://theforeman.org/puppet https://github.com/puppetlabs/puppetchef http://www.opscode.com/chef/fuel https://fuel.mirantis.com/https://github.com/tuskar/tuskarVagrant https://github.com/aodn/vagrant-openstack http://www.cloudsoftcorp.com/blog/getting-started-with-heat-devstack-vagrant/

Review GItopenstack uses https://code.google.com/p/gerrit/wiki/ShowCases

RDO: easy to installRDO -> Enterprisehttp://www.redhat.com/support/subscription-benefits/http://www.redhat.com/about/subscriptionhttp://www.redhat.com/products/jbossenterprisemiddleware/community-enterprise/others:http://openstack.redhat.com/Quickstarthttp://www.youtube.com/watch?v=OsQJmipzBYI

Info geral:http://openstack.redhat.com/Docshttp://www.openstack.org/https://www.openstack.org/joinhttp://www.openstack.org/community/ -> events,jobs,https://wiki.openstack.org/wiki/Main_Pagehttp://docs.openstack.org/http://www.openstack.org/blog/http://devstack.org/http://www.trystack.org/http://www.turnkeylinux.org/https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide/tree/masterhttps://wiki.openstack.org/wiki/DevQuickstarthttp://docs.openstack.org/trunk/openstack-ops/content/http://docs.openstack.org/grizzly/openstack-compute/install/apt/content/conf-files.html

http://ilearnstack.com/openstack/


Questions?


Extra: more about OpenStack Architecture and how to scale

From Russell Bryant (Red Hat Summit)



Compute

Dashboard

ImageService

Identity

ObjectStorage



Mention code names, each project has official and code name

Code names shorter, used in code, used in conversation, mixed usage in docs (talk about Identity, run the keystone command)



● Important Concepts● Tenants – Groups of Users● Users● Roles● Tokens● Services

Tenants – Groups of users (could be a team, a whole company, whatever you want), share resources

Users - ... users

Roles - admin or not is all it's used for so far

Tokens – auth to get a token. Token allows access to all other OpenStack APIs.



keystone

Token ServicesToken Identity

ReST API

Presents a REST API (most services do)

Separate backends for tokens, identity, and services

Tokens – db, memcache

Identity – db, LDAP

Services – db, flat file backed


keystone...

Load Balancer

keystone keystone

OpenStack Identity (Keystone) Scaling

Horizontal Scaling with a load balancer

This pattern will be seen all over OpenStack. HAProxy is a software load balancer that we support.

All services use the same storage backend (if MySQL, use strategies to scale MySQL)



Compute

Dashboard

ImageService

Identity

ObjectStorage



Web Dashboard – self-service UI for creating and managing your own compute, networking, and storage resources

Start off by showing some of the screens


OpenStack APIs


● Python WSGI Application

● Django framework

● Stateless client of OpenStack APIs

httpd

horizon


OpenStack APIs

OpenStack Dashboard (Horizon) Scaling

httpd

horizon

httpd

horizon

httpd

horizon...

Load Balancer

Session Storage

Horizontally scale out httpd w/ a load balancer

All horizon instances use the same OpenStack APIs

What's different: Session state shared via Django's session engine, so whichever backends supported there

default is local cache

others are memcached, db, db+caching, signed cookies



Compute

Dashboard

ImageService

Identity

ObjectStorage



Another clever code name - “Heat keeps the clouds up”



● Consumes all other OpenStack APIs

● Important concept: template defined stacks

Why Heat is awesome: Template defines your application in terms of all of the resources needed (instances, networks, database, load balancer)

Version your deployment like your software

Repeatable complex deployments that are fully automated

AWS Cloudformation template compatible, but with added OpenStack resources

AWS Cloudformation APIAlso has an OpenStack API, native template syntax in the works

Implements HA, auto scaling, was shown in keynote this AM


OpenStack APIs


heat-api

ReST API / CFN API

AMQP

heat-engine

DB

Heat-api – serves up the ReST API and/or CFN API

First time AMQP comes up. Explain it. Advanced Message Queueing Protocol. Scalable messaging between applications. We use Qpid.

Api talks to engine via AMQP

Engine does the real work of setting up the stack. Makes many API calls to other OpenStack services to set up all of the resources defined in the template

More about AMQP http://docs.openstack.org/developer/nova/devref/rpc.html


OpenStack APIs

OpenStack Orchestration (Heat) Scaling

heat-api

AMQP

heat-engine

heat-apiheat-api ...

Load Balancer

heat-engineheat-engine ...

Horizontal scaling like others

What's different: AMQP distribution, db per heat-engine



Compute

Dashboard

ImageService

Identity

ObjectStorage





● The foundation of billing or charge back systems

● Concepts● Meters● Compute Pollsters● Central Pollster● Notifications● Collectors

Meters – any type of usage data, transformed into billable items by your own system

How much of a thing, how long, whatever makes sense (CPU hours, bandwidth)

Notifications – Most projects emit usage notifications via AMQP

Compute pollsters – poll for other data on compute nodes

Central pollster – plug point, poll for data from elsewhere

Collectors – collect meters from all of these places and store it (in a db, mongodb, mysql/postgres)



Credit: Doug Hellmanhttp://stevedore.readthedocs.org/en/latest/essays/pycon2013.html#requirements-for-ceilometer

Really nice conceptual architecture diagram



Another view of the architecture, this one showing actual services that run, including the API service

A few services funneling meters to the collector

Both collector and API access the backend store

Talk about scaling from this slide:

Horizontal scaling of API, compute pollsters, collector



Compute

Dashboard

ImageService

Identity

ObjectStorage





● Concepts● Accounts● Containers – Organize Your Data● Objects – Your Data● Ring – Internal Data Structure

Object Storage ... store and retrieve blobs of data

Note that swift manages replicas of data across multiple storage nodes

Ring internal, basically a distributed hash table, but exposed to you as an admin. You tell swift info about your deployment so it can build the ring



Credit: Mark McLoughlin

API

Storage

Basically two node types, API nodes and storage nodes

Storage nodes hold a subset of data, one of the replicas of it

How does scaling work with these 2 node types?


Proxy Proxy Proxy...

Load balancer

Storage Storage Storage...

OpenStack Object Storage (Swift) Scaling

Horizontal scaling like the rest

What's different here is the storage nodes: can run as many as you need, the ring is generated to account for added/removed nodes, rebalancing done as needed



Compute

Dashboard

ImageService

Identity

ObjectStorage





● Concepts● Images● Metadata● Storage Backends

Registry for Images – disk images for creating VMs

Example metadata: - name of the image - size - access control (public or private?) - hardware info, hw_vif_driver=e1000 vs virtio

Storage backends - filesystem backed (local, or glusterfs) - Swift



glance-api

glance-registry

DBImage Storage

ReST API

ReST


OpenStack Image Service (Glance) Scaling

glance-api

glance-registry

DBImage Storage

glance-apiglance-api ...

Load Balancer

* Scales horizontally the same way as the API

Horizontal scaling of the api as usual

All accessing the same backend image storage

Glance-registry simplified for the diagram, but can horizontally scale with a load balancer as well



Compute

Dashboard

ImageService

Identity

ObjectStorage





● Concepts● Volumes● Snapshots● Storage Backends

Volumes can be hotplugged to running Vms, but that's handled by the compute service

Storage backends - LVM based storage the default, iSCSI - Various storage vendors' appliances, iSCSI - NFS, file backed - GlusterFS, file backed



cinder-api

ReST API

AMQP

cinder-volume

cinder-scheduler

DB

ReST API for external, AMQP internal, as seen before

Scheduler for placement logic

Volume servers for managing storage


cinder-schedulercinder-scheduler

OpenStack Block Storage (Cinder) Scaling

cinder-api

AMQP

cinder-volume

cinder-scheduler

cinder-volumecinder-volume

cinder-api cinder-api...

Load Balancer

Horizontal scaling using patterns seen before



Compute

Dashboard

ImageService

Identity

ObjectStorage



Plugin is networking tech specific, Not all have agents

Horizontal Scaling of the API

L2 agents – for some, not all plugins

DHCP agents – IP address management, networks scheduled to DHCP agents

L3 agents – can create routers to connect multiple networks, gateway to external networks, floating IP support, security groups, scale by scheduling routers

New stuff happening: LbaaS and VPNaaS

Scale: horizontal API, L2 agents run on all nodes, can run multiple DHCP agents and L3 agents



● Concepts● Networks● Routers● Subnets● Ports● Vendor plugins

Network. An isolated virtual layer-2 domain. Virtual networks for your Vms, or provider networks that map to physical networks in your data center

Routers – Connectivity to external networks, connect multiple virtual networks together

Subnet. An IP address block. IPs assigned to ports.

Port. A virtual, or logical, switch port on a specified network. Instances get attached to these.

Like Cinder, Quantum has lots of plugins for various vendors' networking technologies

The default open source stuff: linux bridges, Openvswitch



Network. An isolated virtual layer-2 domain. Virtual networks for your Vms, or provider networks that map to physical networks in your data center

Routers – Connectivity to external networks, connect multiple virtual networks together

Subnet. An IP address block. IPs assigned to ports.

Port. A virtual, or logical, switch port on a specified network. Instances get attached to these.

Like Cinder, Quantum has lots of plugins for various vendors' networking technologies

The default open source stuff: linux bridges, Openvswitch



Compute

Dashboard

ImageService

Identity

ObjectStorage



My favorite because I'm biased



● Concepts● Instances / Servers● Flavors / Instance Types● Virt drivers

● OpenStack API and EC2 API

Instances – Vms

Flavors – resources (CPUs, RAM, disk)

Virt drivers – choice of hypervisor, KVM most popular, even supports bare metal provisioning!

Native OpenStack ReST APIEC2 API



nova-api

AMQP

nova-compute

Libvirt+KVM

DB

nova-scheduler

nova-conductor

ReST API

Looks very similar to Cinder, (because Cinder came from Nova)

Api, scheduler, compute for managing hypervisor nodes

What's different: nova-conductor

For security reasons, want to isolate compute nodes as much as possible, so no direct db access



AMQP

Libvirt+KVM

DB

nova-scheduler

nova-conductor


nova-compute

nova-api

Load Balancer

This works ... to a point (can still support many thousands of nodes)

DB and message broker become a pain point

There's another level scaling being worked on now: cells, will show you how cells works, starting with this picture



API Cell


AMQP

Cells, each with their own message broker and database

Communication over AMQP between the nova-cells service in each cell

Cells can be local or geographically distributed, all under a single API endpoint

Next diagrams show what's in a cell



AMQP

Libvirt+KVM

DB

nova-scheduler

nova-conductor


nova-compute

Compute Cellnova-cells

Removed nova-api, added nova-cells service

Its own db and message broker



AMQP

DB

API Cell

nova-cells

nova-api

Load Balancer

ReST API

Removed everything but nova-api, added nova-cells service



API Cell


AMQP

Again, this is what it looks like

Large amounts of nodes grouped into cells, federated using AMQP


OpenStack Components

Compute

Dashboard

ImageService

Identity

ObjectStorage



Deep breath!

Now that we've taken the deep dive, let's step back out

Performing operations in compute often requires interacting with multiple services

Now will go through an example (multiple slides), start an instance

Start an instance – auth with keystone, request instance from nova, get image from glance, potentially attach volume from cinder, get network info from quantum, fire off notifications consumed by ceilometer



Compute

Dashboard

ImageService

Identity

ObjectStorage


NetworkingBlock

Storage

1

Step 1 – get a token

Single keystone service, stores token in its token db



Compute

Dashboard

ImageService

Identity

ObjectStorage


NetworkingBlock

Storage

1 2

Step 2 – Request server from compute service

Next slide has nova diagram to talk about how it gets processed



nova-api

AMQP

nova-compute

Libvirt+KVM

DB

nova-scheduler

nova-conductor

ReST API

Boot a server

API -> scheduler -> compute



Compute

Dashboard

ImageService

Identity

ObjectStorage


NetworkingBlock

Storage

1 2

3

Step 3 – Download image to compue node if necessary

Talks to the glance-api service, image data streamed from storage backend



Compute

Dashboard

ImageService

Identity

ObjectStorage


NetworkingBlock

Storage

1 2

3

4

Step 4 –Request block storage volume info so that it can be attached to an instance

Talks to the cinder-api service



Compute

Dashboard

ImageService

Identity

ObjectStorage


NetworkingBlock

Storage

1 2

3

4

5

Step 5 –Request network information

Talks to quantum-api service, tell it the network(s) that were requested, quantum will allocate port



Compute

Dashboard

ImageService

Identity

ObjectStorage


NetworkingBlock

Storage

1 2

3

4

5

6

Step 6 –With all info needed, boot the VM

Back to the nova-compute service, boots the VM by giving libvirt all necessary info

Open stack

Documents

Transcript of Open stack