Optimizing Cloud Foundry and OpenStack for large scale deployments

download Optimizing Cloud Foundry and OpenStack for large scale deployments

If you can't read please download the document

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Optimizing Cloud Foundry and OpenStack for large scale deployments

Cloud Foundry

Optimizing OpenStack for large scale Cloud Foundry deployments

Animesh Singh, Daniel Krook, Jason Anderson, Kalonji Bankole@animeshsingh @danielkrook @andersonljason @k_bankole

#OpenStack Summit May 12-16, 2014 Atlanta, Georgia

# 2014 IBM Corporation



Infrastructureas a Service

Platformas a Service

Softwareas a Service

API economy



TOSCAhttp://www.ibm.com/developerworks/cloud/library/cl-open-architecture/IBM cloud and open technologiesAt all tiers, IBM is committed to building its cloud on an open cloud architectureProducts and services built on open source and open standards benefit IBM and customers

# 2014 IBM CorporationCloud computing needs to be built on open source and standards. To build flexible and interoperable cloud architectures, IBM is building on open cloud projects - OpenStack for IaaS, deployment automation tools like Chef, PaaS fabrics like Cloud Foundry , databases like MongoDB, portable workload template standards like TOSCA and many others.

Gold Sponsors

IBM is #2 in contributions to integrated projects IBM is working to accelerate OpenStack Foundation successBecause an open interoperable Cloud is critical for flexible cloud deployment and customer success IBM has 15 core contributors15 IBMers working on OpenStack from formation of the Foundation to Code Quality & New Function380 IBMers active in the projects2107

Mar2013May2014859Contributors8,500 Individuals2130Contributors16,100+ IndividualsExponential growth

Platinum SponsorsOpenStack Participant Growth

# 2014 IBM CorporationIBM is committed to accelerating the success of the OpenStack foundation because interoperability in the Cloud is critical for flexible cloud deployments and ultimately customer success. As a member of the new Board of Directors, our goal is to help progress the platform, sustain a vibrant ecosystem, and position OpenStack as the IaaS platform of choice for cloud consumers and providers.

IBM made a commitment in March 2013 that ALL of our cloud offerings would be based on OpenStack), including our private clous offerinds to our public cloud offerings (currently Softlayer), our optimized Pure Systems offerings (PureApp and PureFlex, etc) : which would be transitioned over time to be hosted on OpenStack.

Out private and public cloud portfolios including Softlayer and IBM Cloud Manager are already incorporating these standards.


42014 IBM CorporationAnd embracing Cloud Foundry as an open source PaaSContinuing our history of strengthening and extending open source



1,274Average per month2013 average: 13312mo average: 98Lifetime average: 58711k50+43

# 2014 IBM CorporationCloud Foundry PaaS

Cloud Foundry services registry and runtime management layer.

Components are dynamically discoverable and loosely coupled, exposing health through HTTP endpoints so agents cancollect state and act on it.

Cloud Foundry architectural overviewUser Authentication and Authorization


DEA Pool Service GatewayApps

Service Connector

Health Manager


Cloud Controller

Build Packs

Cloud Foundry BOSH




Service Nodes

# 2014 IBM CorporationCloud Foundry PaaSAn application runs in a DEA. The Cloud Controller orchestrates the routing and lifecycle of all DEAs in the pool. Routers manage application traffic. Health Manager reports mismatched application states to the CC. A service gateway provides an interface for services (native or external). A messaging bus manages all system communication. Apps are accessed directly through the router while web and CLI clients access Cloud Controller via RESTful services.5


# 2014 IBM CorporationStock photos for IBMhttps://www-304.ibm.com/connections/wikis/home?lang=en-us#!/wiki/Wc2b28dd4ba19_4a82_a6c1_8aaee8ffc00a/page/Photography6


BOSH Cloud Provider Interface (CPI) Initially written for VMware and Amazon Web Services OpenStack CPI OpenStack CPI written by PistonCloud Apache2 license PistonCloud - An enterprise OpenStack company, founded by few of OpenStack foundersCloud Foundry OpenStack integration









# 2014 IBM CorporationHere we show the stack for deploying a Platform as a Service. At the very bottom is hardware layer comprising of servers, networking and storage, and then comes the IaaS layer , in thiscase OpenStack. PaaS sits at the very top - though we need to have a layer which integrates PaaS to the underlying IaaS , which in this case is BOSH Cloud Provider Interface.

BOSH is release deployment tool for Cloud Foundry, and it has CPIs written for VMWare, AWS and OpenStack. BOSH CPI has few of the methods which need to be implemented for anyIaaS, and in this case have been implemented for OpenStack and leverage the preexisting fog ruby gem for talking to OpenStack APIs

BOSH deployment process

Deployment ManifestRelease name/version# VMs, job paramsStemcells to useStemcellBase OSBOSH agentReleaseName

JobsSoftware packagesConfig templatesScriptsBOSHDeployedEnvironment

Virtual MachineConfigurationSoftware PackagesVirtual MachineConfigurationSoftware PackagesVirtual MachineConfigurationSoftware PackagesVirtual MachineConfigurationSoftware packages

# 2014 IBM CorporationHere we introduce some of the BOSH vocbulary as we will be using it in the course of the presentation.

Stemcells: In a cloud platform, VMs are usually cloned from a template. A stemcell is a VM template containing a standard Ubuntu distribution. A BOSH agent is also embedded in the template so that BOSH can take control of VMs cloned from the stemcell.

Jobs: A job is a collection of software which serves a particular purpose (e.g. MySQL or the Cloud Controller). At deployment time, each job will be install on its own stemcell VM.

Releases: A release contains a number of jobs which can be deployed into the target environment. A deployment can consist of more than one release and not every job in a release must be deployed.

Deployment Manifest: A deployment manifest is the set of instructions BOSH uses to create a deployment. It is written using the YAML format. The manifest file contains the following sections:NameReleasesCompilationUpdateNetworksResource PoolsJobsProperties

Sample WordPress Manifest FileSample Cloud Foundry Manifest File


Sample manifest

# 2014 IBM Corporation10Cloud Provider Interface for OpenStack OpenStack CPI is an implementation of the BOSH CPI.

Leverages the Fog Ruby gem for OpenStack

OpenStack CPI gem (Ruby package), the OpenStack Registry and Agents all reside on the Cloud Foundry Bosh repository https://github.com/cloudfoundry/bosh

Stemcell Image (Base OS Image) and sample manifest files for OpenStack exist

# 2014 IBM Corporation

Requirements:Static / floating ipsPersistent disksOutbound Internet connectivityCustom flavorsIncreased quotaSecurity groups

11Integration requirements for OpenStack

# 2014 IBM CorporationNow delving more into technical details - we first need to have an OpenStack environment implmented and configured accroding to some of the requirements from Cloud Foundry, for example static ips and persistent disks, as well as the VMs should have connectivity to the internet.

We also need to create some custom flavors for Cloud Foundry jobs, as well as the default scheduler for OpenStack needs to be set up so that it distributes VMs across different nodes in a random order.



# 2014 IBM CorporationStock photos for IBMhttps://www-304.ibm.com/connections/wikis/home?lang=en-us#!/wiki/Wc2b28dd4ba19_4a82_a6c1_8aaee8ffc00a/page/Photography


OpenStack discovery:

Extend the Fog OpenStack gem used by CPI to discover and OpenStack artifacts in an automated manner in the Cloud Foundry manifestPass OpenStack credentials and discover OpenStack compute and network information.

Deployment automation: Tip one

Discover FlavorsDiscover NetworksDiscover VM SubnetDiscover DHCP IPDiscover KeypairsDiscover Gateway IPDiscover Security groups

# 2014 IBM CorporationDeployment automation: Tip two

OpenStack pre-req setup:

Extend the Fog OpenStack gem used by CPI to setup Cloud Foundry OpenStack pre-reqs Setup tenant quota Keypairs Custom Flavors Security Groups setup

Create KeypairCreate FlavorsRouter, DEA, Cloud Controller, Service Nodes Create Security GroupsCf-private, cf-publicSetup tenant quota

# 2014 IBM CorporationDeployment automation: Tip three

Cloud Foundry manifest generation:

Hard to manually create and edit Cloud Foundry manifests which can extend up to 1000+ lines Automation around template creation an extension from the previous efforts of OpenStack Discovery and Cloud Foundry pre-req setup. Can create templates for both nova-network and neutron based environmentsIterates through jobs and assigns static IPs to them if we are using an environment without PowerDNS.

# 2014 IBM Corporation


Deployment automation: Tip four

Stemcell automation:

Community stemcell image has empty fstab and this makes the root partition auto detection to fail, and consequently provisioning to fail.

Multiple Solutions: Inject file