Cloud foundry elastic architecture and deploy based on openstack
-
Upload
open-source-software-competence-center-huawei-technologies -
Category
Technology
-
view
1.159 -
download
5
Transcript of Cloud foundry elastic architecture and deploy based on openstack
1© Copyright 2012 EMC Corporation. All rights reserved.
CloudFoundry: Elastic Architecture Design and Deploy Based on OpenStack
郭峰 (Golfen Guo)@ 教授也是浮云[email protected]
Cloud Platform and Application Lab, EMC Labs China
2© Copyright 2012 EMC Corporation. All rights reserved.
EMC Labs China
Advanced Technology Research and Development
Big Data Lab
Cloud Infrastructure and System Lab
Cloud Platform and Applications Lab
University Collaboration
Industry Standards Office
IP Portfolio Development
VisionBecome an elite research and
advanced technology institute in China
-Become the model
for future EMC Labs worldwide
3© Copyright 2012 EMC Corporation. All rights reserved.
Agenda
• What is PaaS
• CloudFoundry Overview
• Elastic Architecture in CloudFoundry
• Introduce to BOSH
• CPI and OpenStack
• Deploy PaaS with BOSH
4© Copyright 2012 EMC Corporation. All rights reserved.
What is PaaS? (from Search Cloud Computing)
• Platform as a Service (PaaS) is a way to rent hardware, operating systems, storage and network capacity over the Internet.
• The service delivery model allows the customer to rent virtualized servers and associated services for running existing applications or developing and testing new ones.
RENT, not BUILD one!
Virtual Server and Associated Service
5© Copyright 2012 EMC Corporation. All rights reserved.
How PaaS Benefit Us?
Before: Now:
6© Copyright 2012 EMC Corporation. All rights reserved.
How PaaS Benefit Us? (Cont.)
Traditional IT
Storage
Servers
Networking
O/S
Middleware
Virtualization
Data
Applications
Runtime
You
Man
age
O/S
PaaS
Storage
Servers
Networking
Middleware
Virtualization
Data
Applications
Runtime
You
Man
age
Man
ag
ed b
y P
latfo
rm
Agility and Cost
Savings
O/S
IaaS
Storage
Servers
Networking
Middleware
Virtualization
Data
Applications
RuntimeY
ou M
anag
eM
an
ag
ed b
y P
latfo
rm
From Accelerating your Journey to Application Transformation, EMC World 2012
PaaS powered
7© Copyright 2012 EMC Corporation. All rights reserved.
Cloud Foundry – The Open PaaS
8© Copyright 2012 EMC Corporation. All rights reserved.
PaaS in Action…
$ cat hw.c#include <stdio.h>main() { printf(“Hello World\n”);}
$ cc hw.c; ./a.out
Another analogy from Mark Lucovsky,
Every one knows, “Hello World”:
9© Copyright 2012 EMC Corporation. All rights reserved.
PaaS in Action... (Cont. )
$ cat hw.rb require 'rubygems' require 'sinatra'
$hits = 0 get '/' do $hits = $hits + 1 "Hello World - #{$hits}" end
$ vmc push hw
“Hello World” in the Cloud:
10© Copyright 2012 EMC Corporation. All rights reserved.
Scale App On Demand
Step 1: Scale instance to 10:
$ vmc instances hw 10
Step 2: Change the codes:
get '/' do $hits = $redis.incr(‘hits’) "Hello World - #{$hits}" end
Redis
Step 3: bind Redis service to App:
$ vmc create-service redis –bind hw
Implement a Scalability “Hello World” in Cloud:
11© Copyright 2012 EMC Corporation. All rights reserved.
Scale App On Demand(Cont.)
From Cloud Foundry Launch Event, April, 2011 PaaS powered
12© Copyright 2012 EMC Corporation. All rights reserved.
Simplify Deployment [mysqld]user = foobarport = 3306basedir = /usrbind-address = 172.58.77.101key_buffer = 16Mthread_stack = 128Kthread_cache_size = 8…
[nginx]http.include mime.types;default_type: application/octet-stream;log_format: main ‘$remote_addr - $remote_user []…’keepalive_timeout 65;
[tomcat]<Connector redirectPort=“8443” emptySessionPath…/><bean id=“sessionFactory” class=“org.springframework…/> [frontend]dependencies:- mysqlclient- rubyfiles:- core/app/fe/**/*- core/common/**/*
[blah]- blah blah blah
Web App
Before:
13© Copyright 2012 EMC Corporation. All rights reserved.
Simplify Deployment (Cont.)
Web App
# to target and login to cloud foundryvmc target http://api.cloudfoundry.comvmc login
# to create and boot the app for the first timevmc push myapp –instances 2 –mem 64M –path ../code
# to create the database and bind it to the appvmc create-service mysql –name mydb –bind myapp
# update live app with new codevmc update myapp –path ../code
Now:
14© Copyright 2012 EMC Corporation. All rights reserved.
First Touch CloudFoundry
prompt> gem install vmcprompt> vmc target api.cloudfoundry.comprompt> vmc loginprompt> vmc push
Would you like to deploy from the current directory? [Yn] Yes Application Name: hello Application Deployed URL: 'hello.cloudfoundry.com'? hello-bob.cloudfoundry.com Detected a Sinatra Application, is this correct? [Yn] Yes Memory Reservation [Default:128M] (64M, 128M, 256M, 512M or 1G) (Press Enter to take default) Would you like to bind any services to 'hello'? [yN]: No Uploading Application: Checking for available resources: OK Packing application: OK Uploading (0K): OK Push Status: OK Staging Application: OK Starting Application: OK
15© Copyright 2012 EMC Corporation. All rights reserved.
First Touch CloudFoundry(Cont.)Upload your app
Your apps deployed
Your all created services
Memory of each instance
Change served instances
Start, stop, update, restart
Services of current app
Information of app
16© Copyright 2012 EMC Corporation. All rights reserved.
PaaS Architecture Pattern
RabbitMQ
Redis
MongoDB
MySQL
17© Copyright 2012 EMC Corporation. All rights reserved.
PaaS Architecture Pattern (Cont. )
18© Copyright 2012 EMC Corporation. All rights reserved.
PaaS Architecture Pattern (Cont. )
19© Copyright 2012 EMC Corporation. All rights reserved.
Conclusion
Abstract to three layers:
• Routers for finding right endpoint of Apps
• Runtime for Apps
• Services provided by platform, consumed by Apps
20© Copyright 2012 EMC Corporation. All rights reserved.
The Keys of Design…
• Elastic
• Failover/System Robust
• Resource Recycling
21© Copyright 2012 EMC Corporation. All rights reserved.
Elastic Architecture in CloudFoundry
Design principals:1. Each components can be run standalone;2. Each components can be scale-out, and notify the peers with
message;3. The components communicate only with message or RESTful
API.
Self-government and Loose Couples :• Easy to add new components. eg Stager, UAA, ACM…;• Easy to evolve each component. eg. CC_ng, Router v2…
22© Copyright 2012 EMC Corporation. All rights reserved.
Elastic Architecture in CloudFoundryA request comes…
23© Copyright 2012 EMC Corporation. All rights reserved.
Pluggable Runtime Support
Refers to https://github.com/cloudfoundry/vcap-staging
• stager -> vcap-staging
Stager StagingPlugin
PHP Rails3Java_We
bPlay …
klass = StagingPlugin.load_plugin_for(plugin_name)plugin = klass.from_file(config_path)plugin.stage_application
extends StagingPlugin
Start Script
Stop Scrip
tServe
r
App
24© Copyright 2012 EMC Corporation. All rights reserved.
Pluggable Runtime Support (Cont.)
So what we need to do is…• Extends Class StagingPlugin in
Common.rb
Line3 ~ Line 62, 59 lines of codes to support PHP.
25© Copyright 2012 EMC Corporation. All rights reserved.
Pluggable Runtime Support (Cont.)
Key methods to rewrite:• stage_application
• start_command
• startup_script
• stop_command
• stop_script
26© Copyright 2012 EMC Corporation. All rights reserved.
Pluggable Services SupportRefers to a nice presentation by Nicholas
Kushmerick
Cloud Foundry Services in last forum:• Service advertisement
• Service Gateway -> Cloud Controller
• POST /services/v1/offerings
• DELETE /services/v1/offerings/:label
• Instance management
• Cloud Controller -> Service Gateway
• Provision: POST /gateway/v1/configurations
• Bind: POST /gateway/v1/configurations/:id/handles
• Unbind: DELETE /gateway/v1/configurations/:id/handles/:handle
• Unprovision: DELETE /gateway/v1/configurations/:id
27© Copyright 2012 EMC Corporation. All rights reserved.
But …
• Modular Design
• Version Tolerance
• Flexible Runtime/Service
• Elastic Architecture
• Many kinds of nodes
• Many nodes each kind
Complex deployment process like other distribution system
Trade Off
28© Copyright 2012 EMC Corporation. All rights reserved.
Practical problem at CloudFoundry.com
40+ unique node types
75+ unique software packages
500-5,000 VMs
2x/week cf.com updates
24*7*365 non-stop operation
No-downtime deployments
Reliable ,robust ,repeatable deployments ,updates , capacity adjustments
Small teams manage many instances
29© Copyright 2012 EMC Corporation. All rights reserved.
How to deploy CloudFoundry?
30© Copyright 2012 EMC Corporation. All rights reserved.
CloudFoundry BOSH CloudFoundry BOSH is an open source tool-chain for release
engineering, deployment, and lifecycle management of large scale distributed services
– Prescriptive way of creating releases and managing systems and services
– It is not a collection of shell scripts, not a pile of Perl
Built to deploy and manage production-class, large scale clusters– Production grade Cloud Foundry clusters: 500+ VMs, 40+ jobs, 75+
packages– Multi-node, multi-tier, complex clusters
Built for DevOps usage and scale by a crack team of veterans– A project, not a product: command line interface, YAML, etc.– Continuous improvement, iterative development, rough edges
https://github.com/cloudfoundry/bosh
31© Copyright 2012 EMC Corporation. All rights reserved.
CloudFoundry BOSH (Cont.)
Built from the need to operate cloudfoundry.com– Over 40 different types of VMs– Hundreds of running instances of VMs– Pushing at least 2 changes per week
End-to-end management– Release engineering– Deployment– Lifecycle management
Generic solution– Any service– Any cloud– Any operating system
32© Copyright 2012 EMC Corporation. All rights reserved.
BOSH Architecture
1. Upload Stemcell1. CLI -> Director2. Director -> Blobstore
2. Upload Release1. CLI -> Director2. Director -> Blobstore
3. Deploy1. CLI -> Director2. Director -> A3. Agent -> B4. Agent -> C
33© Copyright 2012 EMC Corporation. All rights reserved.
BOSH Concepts Stemcell
– VM template– BOSH Agent– IaaS Plugin
Release– Jobs
Job– Packages– Templates (scripts, confs)– Monitoring
Package– Source/blobs– Dependencies– Packaging (scripts)
34© Copyright 2012 EMC Corporation. All rights reserved.
IaaS Neutral
Cloud Foundry BOSH
Cloud Provider Interface(CPI)
vSphere: battle tested implement AWS: code complete
https://github.com/piston/openstack-bosh-cpi
OpenStack: testable release
35© Copyright 2012 EMC Corporation. All rights reserved.
Cloud Provider Interface
Stemcell– create_stemcell (image, cloud_properties)– delete_stemcell (stemcell)
VM– create_vm (agent_id, stemcell, resource_pool, networks,
disk_locality, env)– delete_vm (vm)– reboot_vm (vm)– configure_networks (vm, networks)
Disk– create_disk (size, vm_locality)– delete_disk (disk)– attach_disk (vm, disk)– detach_disk (vm, disk)
36© Copyright 2012 EMC Corporation. All rights reserved.
VM
Cloud Provider Interface (Cont.)For OpenStack
Stemcell– create_stemcell– delete_stemcell
VM– create_vm– delete_vm– reboot_vm– configure_networks
Disk– create_disk– delete_disk– attach_disk– detach_disk
Cloud Provider Interface
…
OpenStack Compute
OpenStackImage Service
VMVM
VMVM
37© Copyright 2012 EMC Corporation. All rights reserved.
Deploy PaaS with BOSH
IaaS
BOSH
CLI
38© Copyright 2012 EMC Corporation. All rights reserved.
Deployments
Release
Network
Resource pools
Jobs
Properties
Update concurrency
Compilation workers
Cloud properties
39© Copyright 2012 EMC Corporation. All rights reserved.
Deployments for CloudFoundyCloudfoundry.ymlname: cloudfoundry
release: name: cloudfoundry version: 89.1-dev
compilation: workers: 4 network: default cloud_properties: ram: 1024 disk: 2048 cpu: 2
update: canaries: 1 canary_watch_time: 3000-90000 update_watch_time: 3000-90000 max_in_flight: 2 max_errors: 1
40© Copyright 2012 EMC Corporation. All rights reserved.
Deployments for CloudFoundy (Cont.)Cloudfoundry.yml
networks: - name: default subnets: - static: - 192.168.2.50 - 192.168.2.89 range: 192.168.2.0/24 gateway: 192.168.2.1 dns: - 10.254.174.10 cloud_properties: name: PrivateNetwork- name: lb subnets: - static: - 192.168.2.90 - 192.168.2.99 range: 192.168.2.0/24 gateway: 192.168.2.1 dns: - 10.254.174.10 cloud_properties: name: PrivateNetwork
41© Copyright 2012 EMC Corporation. All rights reserved.
Deployments for CloudFoundy (Cont.)Cloudfoundry.ymlresource_pools:
- name: infrastructure network: default size: 29 stemcell: name: bosh-stemcell version: 0.6.2 cloud_properties: ram: 256 disk: 2048 cpu: 1 env: bosh: password:
42© Copyright 2012 EMC Corporation. All rights reserved.
Deployments for CloudFoundy (Cont.)Cloudfoundry.ymljobs:
- name: cloud_controller template: cloud_controller instances: 1 resource_pool: infrastructure networks: - name: default static_ips: - 192.168.2.60
- name: nats template: nats instances: 1 resource_pool: infrastructure networks: - name: default static_ips: - 192.168.2.52
43© Copyright 2012 EMC Corporation. All rights reserved.
Deployments(CloudFoundy)Cloudfoundry.yml
properties: domain: cflocal.com
env: {}
networks: apps: default management: default
nats: user: nats password: aaa3ij3122 address: 192.168.2.52 port: 4222
44© Copyright 2012 EMC Corporation. All rights reserved.
Use Case
bosh deployment cf.yml
git pull
bosh upload releaseRun
bosh deployDone!
bosh target cf
Bug report to Dev
45© Copyright 2012 EMC Corporation. All rights reserved.
Acknowledgments
VMware China R&D Center Network & Information Center, Shanghai Jiao Tong Univ.
CloudFoundry Communityhttp://www.cloudfoundry.org
Piston Communityhttps://github.com/piston/openstack-bosh-cpi
46© Copyright 2012 EMC Corporation. All rights reserved.
Q & A
47© Copyright 2012 EMC Corporation. All rights reserved.
Backup Slides
48© Copyright 2012 EMC Corporation. All rights reserved.
Management Path– vmc info
health manager
service
cloud controller
dea
app app
db
nats
router
49© Copyright 2012 EMC Corporation. All rights reserved.
Deployment Path– vmc push
health manager
service
cloud controller
dea
app app
db
nats
router
50© Copyright 2012 EMC Corporation. All rights reserved.
Data Path– Client Request
health manager
service
cloud controller
dea
app app
db
nats
router
51© Copyright 2012 EMC Corporation. All rights reserved.
Evolving Cloud Foundry - Kernel
stager
Package cache
uaa
v2
v2
v2
v2
caldecott
cloud controller
router
nats health manager
dea
app appservice
db
warden
v2
v2
v2
v2
52© Copyright 2012 EMC Corporation. All rights reserved.
How to write a custom service?
• Public API for advertising services and managing service instances
• Endpoints:– Cloud Controller– Service Gateway
(one per service)• REST / JSON / HTTP(S)
Cloud Controll
er
MySQL Gatewa
yMongoD
BGatewa
y
53© Copyright 2012 EMC Corporation. All rights reserved.
How to write a custom service? (Cont.)
Step 1: Service advertisement
Service Gateway -> Cloud Controller, two methods: POST /services/v1/offerings DELETE /services/v1/offerings/:label
:label => 'mysql-5.1':url => 'http://12.34.56.78:4321',:plans => ['free', 'bronze', 'silver', 'gold']:tags => ['mysql', 'mysql-5.1', 'database']:active => true,:description => 'MySQL database service':acls => { :wildcards => '*@vmware.com', :plans => { :bronze => { :users => ['[email protected]', '[email protected]'] } }}
54© Copyright 2012 EMC Corporation. All rights reserved.
How to write a custom service? (Cont.)
Step 2: Instance management
Cloud Controller -> Service Gateway, four methods:
Provision: POST /gateway/v1/configs Bind: POST /gateway/v1/configs/:id/handles Unbind: DELETE /gateway/v1/configs/:id/handles/:handle Unprovision: DELETE /gateway/v1/configs/:id
55© Copyright 2012 EMC Corporation. All rights reserved.
Private API App
Service Gateway
Service Node(eg, PostgreSQL)
Cloud Controller
vmc, STS
Developer
End Users
Avmc create-service postgresql
BPOST /services/v1/configurations
ECREATE DATABASE abc123CREATE ROLE pqr456
CPOST /gateway/v1/configurations
G{host: 12.34.56.78, port: 3456, db: abc123, user: pqr456}
HVCAP_SERVICES = [postgres: {host: 12.34.56.78, port: 3456, db: abc123, user: pqr456}]
I GET /foo/bar
SELECT *FROM FOO
host 12.34.56.78Service
(eg, PostgreSQL)
DNATS: provision
F{host: 12.34.56.78, port: 3456, db: abc123, user: pqr456}
Reference implementation
56© Copyright 2012 EMC Corporation. All rights reserved.
For legacy service – Service Broker
• Cloud Foundry users may already have existing services• Legacy database that hasn’t / won’t / can’t be moved
CloudFoundry.com• Services not yet supported
• Service Broker exposes an arbitrary legacy service instance to Cloud Foundry so that it can be used just like any other instance
57© Copyright 2012 EMC Corporation. All rights reserved.
For legacy service – Service Broker (Cont.)
AppService Gateway
Core Service(eg, PostgreSQL)
Cloud Controllervmc, STS
Service Broker
Brokered Service(eg, Oracle DB)
Service Broker Admin Tools
Bregister hrdb {host: 87.65.43.21, port: 6543, db: def123, user: stu456}
CPOST /broker/v1/configuration
EPOST /services/v1/configurations
Dvmc create-service hrdb
F
POST /gateway/v1/configurationsG{host: 87.65.43.21, port: 6543, db: def123, user: stu456}
HVCAP_SERVICES = [hrdb: {host: 87.65.43.21, port:6543, db: def123, user: stu456}]
IGET /foo/bar
SELECT *FROM FOO
host 12.34.56.78
ACREATE DATABASE def123CREATE USER stu456
Connection
Details