Download - Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Transcript
Page 1: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

© Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved. 1

Ramiro Salas, @ramirosalas Advisory Architect | Pivotal

Cloud Foundry and OpenStack

Page 2: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Open Source Communities

OpenStack - IaaS est. 2010

Cloud Foundry - PaaS est. 2011

(Free as in Speech, not as in Beer).

Page 3: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration
Page 4: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Networking

Storage

Servers

Virtualization

O/S

Middleware

Runtime

Data

Applications

Traditional

Networking

Storage

Servers

Virtualization

O/S

Middleware

Runtime

Data

Applications

IaaS

Networking

Storage

Servers

Virtualization

O/S

Middleware

Runtime

Data

Applications

PaaS

You

Man

age Yo

u M

anag

e

You

Man

age

What is a PaaS?

Page 5: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Joshua McKenty

Page 6: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Nova v0.1 was written in 17 days

by 5 people.

Page 7: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

OpenStack & Cloud FoundryLicense Apache v2.0Community Vendors, Users, and Developers

Architecture API-based services and message-passing

How are things the same?

Page 8: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Cloud Foundry Foundation

Page 9: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Cloud Foundry Foundation & OpenStack Members

Page 10: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

OpenStack Cloud Foundry

Language Python Go and Ruby

Release Cycle 6 months, integrated 2 weeks, parallel

Governance Dedicated Foundation Linux Foundation project

Installation tools Various BOSH

Communication Hub IRC (#openstack-dev) Mailing lists

Source code & review Gerritt & Private Git GitHub & pull requests

Adoption Mostly OSS trials and dev/test environments

Mostly commercial production deployments

How are things different?

Page 11: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Units of ValueIaaS - OpenStack •VMs •Networks •Volumes • Images •Security Groups, etc.

Users Don’t Care About: •Hypervisors •Real Network Topologies •How is the Storage

Managed •Where are the Images

Stored •What Hardware is Being

Used

Page 12: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Units of ValueIaaS - OpenStack •VMs •Networks •Volumes • Images •Security Groups, etc.

But they still have to care about: • IP addresses •Disk sizes •VM orchestration •OS Lifecycle •HA/DR

Page 13: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

OpenStack ConstructsGive me a VM

instance  =  nova.servers.create(name="test",  image=image,  flavor=flavor,  key_name="mykey")

Give me a Volumevolume  =  create(8192,  snapshot_id=None,  source_volid=None,    name=VolName,  description=“My  Volume”,  volume_type=None,  user_id=None,  project_id=None,  availability_zone=az1,    metadata=None,  imageRef=None)

Similar for Networks, Images, etc.

Page 14: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

• Focuses exclusively on applications • Abstracts resources even further • No IPs - Message queues instead • No middleware configuration - Buildpacks • Scale automatically • All your logs in the same place • Designed for Cloud Native Apps

So a new layer is born: PaaS

Page 15: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Units of ValuePaaS - Cloud Foundry •Applications •Services

•Containers are transparent • Lifecycle is fully managed •System changes are

declarative (manifest.yml) • Front-ends, middleware,

VMs, etc. all abstracted

Apps run on Containers Services run on VMs

Page 16: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Structured vs. Unstructured PaaSUnstructured •DevOps controls every

aspect of the deliverable app • Filesystem •Ports exposed • Layers •Repositories •Orchestration •Dependencies…

Example: Custom-built systems with different pieces like: •Docker •Kubernetes •Mesos…

Page 17: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

But Often, Containers Alone Aren’t Enough…

Page 18: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Structured vs. Unstructured PaaSStructured •Developers only specifies app

instances, services to bind, and memory.

•PaaS takes care of: •Routing •Security • Filesystem •Ports •Scheduling •High Availability, etc…

They Don’t have to Care about the HOW

Page 19: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration
Page 20: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Cloud Foundry is…The world’s leading open source platform-as-a-service. •Supported by dozens of major organizations • Language and framework agnostic •Manages both VMs and containers •Orchestrates both applications and data services

Founded and commercialized by Pivotal Software, Inc. Code donated to Cloud Foundry Foundation in 2015

Page 21: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

An (Overly) Simple View of the World

Applications •Stateless •Run in Containers •Horizontally Scalable •Disposable •No permanent storage

Data Services •Stateful •Run in Virtual Machines •Multi-tenant •Diagonally scalable •Durable storage

Page 22: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

12Factor.netMethodology for building software that:

•Use declarative formats for setup automation, to minimize time and cost for new developers joining the project; •Have a clean contract with the underlying

operating system, offering maximum portability between execution environments; •Are suitable for deployment on modern cloud

platforms, obviating the need for servers and systems administration; •Minimize divergence between development and

production, enabling continuous deployment for maximum agility; •And can scale up without significant changes to

tooling, architecture, or development practices.

Page 23: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

© Copyright 2014 Pivotal. All rights reserved. 23

▪Screen shot of Dev Console – pointing out simple abstraction▪Demo

Page 24: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

PCF Demo: cf push

Page 25: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

© Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved. 25

CF Architecture

Page 26: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Cloud Foundry: Applications and ServicesServices (virtual machines): managed by “BOSH”

Applications (containers): managed by “Runtime”

Page 27: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Provision services, not machines

Enables continuous delivery

Cloud-agnostic view of Platform Ops

Holistic Toolchain for “rule them all"

Eliminate bespoke automation on top of config management

Why BOSH

Page 28: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Blobstore

BOSH

Health Monitor

DBDeploy my Services

Worker VMs

Messaging

Health Manager

Cloud Controller

Target VM

BOSH Director

NATS

IaaS

Ops Manager + BOSH

Page 29: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Pivotal

Pivotal Cloud Foundry ArchitectureEnterprise Cloud Foundry

Service Foo

Broker DB

Service Bar

Broker MQ

Service Baz

Broker etc

Elastic Runtime

DEA DEA

DEA DEA

vCA

BOSH provisioning / configuration / orchestration

AWS OpenStackvSphere

Operations Manager

Developer Console

Enterprise Support

Core Comps

Cloud Controller

Router

Health Manager

…..

Page 30: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration
Page 31: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

OpenStack Integration

BOSH CPI •Can use S3 interfaces for

blobstore (Swift/Ceph) •Uses Glance API to upload

stemcells • Interfaces directly with Nova

(Cinder and Neutron are called via Nova)

•Credentials obtained via Keystone

Page 32: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Push App

http://…

> cf Cloud Controller

Router

Health Manager

DEA Pool (Droplet Execution Agent)

DEA / Cell

Staging Apps

Running Apps

Warden / Garden

Containerization

NATS (message bus)

(API)

Process flow

Page 33: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Orgs, Spaces, Users and Quotas

Page 34: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Logical division within a Pivotal CF install / Foundation.

Each organization has its own users and assigned quota

User permissions / roles are specified per space within an organization

Sub-divided into Spaces

Organizations

Page 35: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Different quota limits (e.g. “small”, “enterprise”, “default”, “runaway”) can be assigned per Organization

Quota defines

• Total Memory • Total # of Services • Total # of Routes

Quotas and Plans

Page 36: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Logical sub-division within an organization

Services and Applications are created / specified per Space

Users authorized at an organization level can have different roles per space

Same Service can have different meanings per space

Spaces

Page 37: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

© Copyright 2014 Pivotal. All rights reserved. 37

Page 38: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

① Upload app bits and metadata

push app

② Create and bind services

③ Stage application

④ Deploy application

⑤ Manage application health

…which we will depict in a moment

Blobstore DB

Cloud Controller Service Broker Node(s)

DEADEA

DEADEA

+ app MD

+ =

Service credentials

Route

Cloud Foundry Elastic Runtime

Overview: Deploying App to Cloud Foundry Runtime

Page 39: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Cloud Foundry Elastic Runtime

Blobstore DB

DEA Detect Compile ReleaseNoYes

System Buildpacks

+ =

Cloud Controller

Router

• Staging*• /bin/detect• /bin/compile• /bin/release

• Configure droplet• Runtime (Ruby/Java/Node/Python)• Container (Tomcat/Websphere/Jetty)• Application (.WAR, .rb, .js, .py)

Stage an Application

Page 40: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Blobstore Cloud Controller

Messaging(NATS)

DEA DEA DEA

Access App

Router Cloud Foundry

Elastic Runtime

Deploying an Application

Page 41: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Buildpacks

Containers

Droplet Execution Agents

Under the Hood

Page 42: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Defines the rules to create a fully-contained execution environment

+ =

A Droplet is a fully self-sufficient, referentially correct package that can be executed in an isolated

environment

App Buildpack Droplet

Buildpacks

Page 43: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Isolated environments within an OS VM that run Droplets according to defined rules

There can be many Containers per OS VM thus increasing VM utilization and density

Containers

Page 44: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

VMs that host Containers and can create/destroy them as needed or ordered

Droplet Execution Agents

Page 45: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

VMs are an inefficient level of isolation

RAM on Machine VM overhead

!

!

!

!

!

!

!

!

!

J2EE Container Overhead

J2EE Container Overhead

J2EE Container Overhead

Why Containers?

Page 46: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Containers + microservices allow denser packing and looser coupling of components

RAM on Machine VM overhead

" " " " " " "

" " " " " " "

" " " " " " "

Why Containers?

Page 47: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

HTTP

HTTP

HTTP

HTTP

HTTP

HTTP

AMQPAMQP

Relational DB

Key/Value Store

Graph DB

Microservice Architecture Made Easier

Page 48: Cloud Foundry and OpenStack - … Foundry and OpenStack. Open Source Communities OpenStack - IaaS est. 2010 ... • No IPs - Message queues instead • No middleware configuration

Documentation: http://docs.cloudfoundry.org

Resources

Meetups: http://cloudfoundry.meetup.com/