Pattern-based Orchestration - IAAS · Pattern-based Orchestration University of Stuttgart Cloud...
Transcript of Pattern-based Orchestration - IAAS · Pattern-based Orchestration University of Stuttgart Cloud...
© 2015 IBM Corporation
Pattern-based Orchestration University of Stuttgart Cloud Course, November 2015
Thomas Spatzier ([email protected])
© 2015 IBM Corporation 2
Agenda!
• Introduction
• Pattern Modeling Concepts
• OpenStack Heat
• Container Orchestration
• Hybrid Orchestration
© 2015 IBM Corporation 3
What is Orchestration?!
Orchestration describes the automated arrangement, coordination, and management of complex computer systems, middleware, and services.
[…] orchestration is often discussed in the context of service-oriented architecture, virtualization, provisioning, Converged Infrastructure and dynamic datacenter topics. Orchestration […] defines the policies and service levels through automated workflows, provisioning, and change management. This creates an application-aligned infrastructure that can be scaled up or down based on the needs of each application. […]
From Wikipedia, the free encyclopedia. h6p://en.wikipedia.org/wiki/Orchestra=on_(compu=ng)
© 2015 IBM Corporation 4
What is a pattern?!
• A structural model of components of an application-aligned infrastructure, incl. relationships between those components (applications, middleware, virtualized infrastructure, …)
• The definition of automation behavior for those components to control their lifecycle state (scripts, executables, …)
• The definition of policies and SLAs to scale the infrastructure according to application needs
• The definition of semantics for component lifecycle states, relationships and policies to derive orchestration flows from the structural model
Server1 Server2
Web Applica0on Database
Storage Volume
Private Network
Load Balancer
Public Network
2..10
© 2015 IBM Corporation 5
Agenda!
• Introduction
• Pattern Modeling Concepts
• OpenStack Heat
• Container Orchestration
• Hybrid Orchestration
© 2015 IBM Corporation 6
Contribu)ng Members
provides the Interoperable Descrip)on and Solu)on Portability for: Applica)ons, their component Services and Ar)facts, including Rela)onships of these services
is an approved OASIS Standard since Nov. 25th 2013: hFp://docs.oasis-‐open.org/tosca/TOSCA/v1.0/os/TOSCA-‐v1.0-‐os.pdf
The TOSCA standard…
PlaJorm and Infrastructure services
Management and Opera)onal behavior of these services
------------
------------
Standards perspective on patterns – OASIS TOSCA!
© 2015 IBM Corporation 7
Contents of a TOSCA service package!
Defini'on of building blocks for services
… along with the implementa'on ar'facts for manageability opera'ons
… and the defini'on of deployment ar'facts for components
… op'onal orchestra'on flows
Service Template
Prop
er0e
s Interfaces
Node Types
Requirements
Capabili0es Topology Template
Rela0onship Template
Node Template
type for
Cloud Service ARchive (CSAR)
-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐
-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐
Scripts
Installables
Images
Defini'on of Service Templates …
… including a Topology Template describing the structure of a service
Interfaces Prop
er0e
s Rela'onship Types Plans
Defini'on of possible links between components
type for
© 2015 IBM Corporation 8
Thought process when defining patterns with TOSCA!
Deployment Artifact Node Type
R
C
Implementation Artifact
Met
a-M
odel
Rela0onship Type
…
“Orchestrator” Deployed Instances
SugarCRMDB [MySQLDatabase]
R
C
MySQL [MySQL]
R
C
SugarCRMApp [WebApplication]
R
R
Tem
plat
es
Proper'es: DB Name: “mysqldb” …
Proper'es: Admin User: myadmin …
Proper'es: Context root: /mycrm …
Proper'es: HTTP Port: 8080 …
Apache [ApacheWebServer]
R
C
start.sh
MySQL Database
R
C start.sh
RPMs: mysql
mysql-server
configure.sh
MySQLDBConnec0on Type
s
MySQLDBContainer Req. LinuxOS Req.
MySQL
R
C
Primary focus for users
© 2015 IBM Corporation 9
Processing of patterns – declarative vs. imperative!
• Declarative processing – “what is the desired state?” • Processing logic is derived from the topology model • Requires clear specification of node and relationship semantics
• Imperative processing – “how to reach a desired state?” • Processing logic is implemented in orchestration workflows (“plans”) • Requires explicit modeling of those workflows • Requires a standardized API to make workflows portable across
implementations
© 2015 IBM Corporation 10
Role of relationships in declarative processing!
NodeType “Database” C
NodeType “Application” R
Lifecycle operations (create, configure, start, …) are used by an orchestrator
to create and manage components
Connector concept Relationship Types declare
what endpoints they can connect, and which
processing logic they inject to establish a link at runtime
Base Relationship Types HostedOn, DependsOn and ConnectsTo define the base
semantics for processing topology models
Rela0onshipType “DBConnec0on”
NodeTemplate “MyDB” C NodeTemplate
“MyApp” R
is of type is of type is of type
• Use defined lifecycle opera0ons to deploy and manage each component • Use base rela0onship types to derive processing order
• Process a host before a hosted component (HostedOn) • Process a provider before a client (DependsOn, ConnectsTo)
• If a component is related to another component, see if rela0onship injects addi0onal processing logic (e.g. pre-‐configure endpoint)
… a preFy mechanical process
© 2015 IBM Corporation 11
How is a topology processed?!• Use base rela0onship types to derive component processing order • First process a host, then process hosted component • First process a component that another component depends on, then process the dependent component
• First process a component that another component connects to, then process the connec0ng component
• For each component • Deploy its Deployment Ar0facts • Invoke lifecycle opera0ons in right sequence (create, configure, start …); there can be no-‐ops
• If a rela0onship contributes logic, inject it into component opera0on invoca0ons
Web 1
Web 2
Web 3
Web 4
Web 5
DB 1
DB 2
DB 3
DB 4
rela'onship injects logic
DB 1
DB 2
DB 3
DB 4
Web 1
Web 2
Web 3
Web 4
create VM
perform base OS config
install and configure hFpd
install and configure php run0me
create VM
perform base OS config
install and configure MySQL
create and configure SugarCRM database
Web 5 install and configure SugarCRM app, configure database endpoint proper0es
run in parallel
wait for step “DB 4” to complete
© 2015 IBM Corporation 12
Some Application
Com
pute
Sto
rage
arch = x86_64 cpus = 2
mem >= 4GB OS = Linux
size >= 10GB
Database
provider = MySQL version >= 5.5
Requirements & Capabilities!
Database Provider
Database
Com
pute
Sto
rage
Requirements can be fulfilled explicitly by other components in the model
Requirements can be fulfilled by the run0me
Requirements/Capabili'es are base for subs'tutability
Some Component
R
Provider A
C
Provider B
C
Provider C
C
OR OR
Some Application
Com
pute
Sto
rage
arch = x86_64 cpus = 2
mem >= 4GB OS = Linux
size >= 10GB
Database
provider = MySQL version >= 5.5
© 2015 IBM Corporation 13
Model composition!
Node Template
Node Template
Service Template 1
Service Template 2
can subs0
tute
boundary defini0ons
Subsystems can be abstracted in some models.
Other models can define details of subsystems.
à separa0on of concern, re-‐use
© 2015 IBM Corporation 14
Usage scenarios for model composition & substitution!
Load Balancer
VM
Web Server
VM
1..*
Web Server
VM
Single node web 'er Scalable web 'er
WebApp
WebTier
App DB
Tomcat MySQL
VM VM
Network Storage
Applica'on Layer
PlaSorm Layer
Infrastructure Layer
Custom workload
Mul0ple op0ons of
middleware deployments
Mul0ple op0ons of
infrastructure configura0ons
Varying deployment op'ons Layering of models
© 2015 IBM Corporation 15
Agenda!
• Introduction
• Pattern Modeling Concepts
• OpenStack Heat
• Container Orchestration
• Hybrid Orchestration
© 2015 IBM Corporation 16
OpenStack Heat!
• A pattern-based orchestration engine on-top of OpenStack APIs
• Consumes templates that declaratively describe a workload consisting of a set of OpenStack resources (e.g. servers, storage volumes, networks)
• Instantiates and manages instances of those workloads (“stacks”) by orchestrating over the respective OpenStack APIs
Server
Network Storage Volume
Server
Network Storage Volume
Template Stack
© 2015 IBM Corporation 17
Heat Orchestration Template (HOT) example!
heat_template_version: 2013-‐05-‐23 parameters: key_name: type: string description: Name of keypair to assign to servers. image: type: string description: Name of image to use for servers. resources: private_net: type: OS::Neutron::Net properties: # properties of private_net ... private_subnet: type: OS::Neutron::Subnet properties: # properties of private_subnet ... server1: type: OS::Nova::Server properties: # properties of server1 ... outputs: server1_private_ip: description: IP address of server1 in private network value: { get_attr: [ server1, networks, private, 0 ] }
configurable parameters for each stack deployment
the set of resources to deploy for each stack
observable outputs of each stack deployment
resource type to be created
© 2015 IBM Corporation 18
Heat software orchestration!
• New concept introduced in OpenStack Icehouse release, enhanced in Juno release • Up to then, only base OpenStack infrastructure
resource support
• SoftwareComponent or SoftwareConfig: • Definition of re-usable orchestration metadata,
incl. reference to automation for a specific piece of software
• SoftwareDeployment: • Instantiation of a software component on a
specific server • Similar to an association class • Includes concrete input values for deployment
and produces output • Can define dependencies on other deployments
Automation (Chef, Puppet,
script, …) Automation (Chef, Puppet,
script, …)
Server “server1”
So`ware Deployment “app_depl”
So`ware Component “app_comp”
Server “server2”
So`ware Deployment “db_depl”
So`ware Component “db_comp”
“Nod
e Type
” “N
ode Template”
© 2015 IBM Corporation 19
Heat software orchestration example!
heat_template_version: 2013-‐05-‐23 parameters: db_name: type: string description: Name of database to create. resources: db_comp: type: OS::Heat::SoftwareComponent properties: inputs: -‐ name: database_name default: my_db configs: -‐ actions: [ CREATE ] tool: script config: { get_file: install_db.sh } -‐ actions: [ UPDATE ] tool: script config: { get_file: reconfigure_db.sh } db_depl: type: OS::Heat::SoftwareDeployment properties: config: { get_resource: db_comp } server: { get_resource: server } input_values: database_name: { get_param: db_name } server: type: OS::Nova::Server # ... details ...
Automation (Chef, Puppet,
script, …)
Server “server”
So`ware Deployment “db_depl”
So`ware Component “db_comp”
Automation (Chef, Puppet,
script, …)
© 2015 IBM Corporation 20
Heat Provider Templates!
heat_template_version: 2013-‐05-‐23 parameters: database_name: type: string description: Name of database to create. database_admin_user: type: string description: Username to set for database admin. resources: db_server: type: My::DB::Server properties: db_name: { get_param: database_name } db_admin_user: { get_param: database_admin_user } outputs: db_server_ip: description: IP address of database server. value: { get_attr: [ db_server, db_server_ip ] }
heat_template_version: 2013-‐05-‐23 parameters: db_name: type: string description: Name of database to create. db_admin_user: type: string description: Name of database admin user. resources: mysql_config: type: OS::Heat::SoftwareConfig properties: # software configuration properties mysql_deployment: type: OS::Heat::SoftwareDeployment properties: config: { get_resource: mysql_config } server: { get_resource: server } server: type: OS::Nova::Server properties: # server properties outputs: db_server_ip: description: IP address of database server. value: { get_attr: [ server, networks, private, 0 ] }
end user template
“provider template”
“implements”
# Heat environment file # mapping of resource type names to provider templates resource_registry: My::DB::Server: file:///my_provider_template.yaml
© 2015 IBM Corporation 21
Heat resource model!
• Resource types used in templates are implemented as Python classes that support a well-defined lifecycle model
• Out-of-the-box resource type implementations for almost all OpenStack resources (OS::Nova::Server, OS::Cinder::Volume, OS::Neutron::Net, …)
• Additional, custom resource type implementations can be installed as plug-ins
• HOT templates (“provider templates”) can serve as implementation of resources types, however finally must resolve to python-implemented resource types
• SoftwareConfig/SoftwareComponent as a “meta resource type” pointing to actual implementation in external automation assets (scripts, …)
class MyResourceType(resources.Resource): properties_schema = { 'foo': properties.Schema( properties.Schema.STRING, _('foo prop description'), default='foo', required=True ), } attributes_schema = { ‘attr_1': 'The first attribute', } def handle_create(self): # creation logic def check_create_complete(self, token): # check creation status for longer running tasks def handle_update(self): # update logic def check_update_complete(self, token): # check update status for longer running tasks # handle and check methods for suspend/resume ... def handle_delete(self): # deletion logic def check_delete_complete(self, token): # check deletion status for longer running tasks
© 2015 IBM Corporation 22
Heat declarative processing!
• Build dependency graph of all resources • Resources can depend on each other
explicitly: depends_on • Resources can depend on each other
through data flow: • get_attr • get_resource
• Process all resources that do not have any dependency – i.e. there can be a lot of parallel processing • Per resource, invoke handle_create
and wait until resource is in state CREATE COMPLETE (check_create_complete)
• Process all resources according to dependencies
heat_template_version: 2013-‐05-‐23 resources: db_comp: type: OS::Heat::SoftwareComponent properties: outputs: -‐ name: db_port configs: # config actions and scripts ... app_comp: type: OS::Heat::SoftwareComponent properties: inputs: -‐ name: db_port configs: # config actions and scripts ... db_depl: type: OS::Heat::SoftwareDeployment properties: config: { get_resource: db_comp } server: { get_resource: server } app_depl: type: OS::Heat::SoftwareDeployment properties: config: { get_resource: app_comp } server: { get_resource: server } input_values: db_port: { get_attr: [ db_depl, db_port ] } server: type: OS::Nova::Server # ... details ...
© 2015 IBM Corporation 23
Heat stack update!
• Heat supports update of already deployed stacks in a declarative way • Create a stack using original template • Get a template describing the created stack (includes input data provided during
stack creation) • Modify current template (add, remove or updates resources) • Invoke heat stack-update with modified template
• Heat compares the updated template against the currently active template and derives the set of resources to be updated, added or removed
• Changes get applied according to the dependency graph among resources • A change to one resource might trigger changes in dependent resources!
• Resources may support in-place update, or may need replacement • Example for in-place update: change maximum size of an AutoScalingGroup
resource • Example for replacement: use of a different image for a server
© 2015 IBM Corporation 24
Heat stack update example!
Server “server1”
So`wareDeployment “my_app”
db_pw: get_aFr: [ my_db, pw ]
Server “server2”
So`wareDeployment “my_db”
pw: mypassw0rd
Server “server1”
So`wareDeployment “my_app”
db_pw: get_aFr: [ my_db, pw ]
Server “server2”
So`wareDeployment “my_db”
pw: otherpassw0rd
Server “server1”
192.168.100.10
So`wareDeployment “my_app”
db_pw: get_aFr: [ my_db, pw ]
Server “server2”
192.168.100.11
So`wareDeployment “my_db”
pw: mypassw0rd
template stack
2So`wareDeployment
“my_db”
pw: otherpassw0rd
1stack-‐create
template-‐show
stack-‐update
© 2015 IBM Corporation 25
Agenda!
• Introduction
• Pattern Modeling Concepts
• OpenStack Heat
• Container Orchestration
• Hybrid Orchestration
© 2015 IBM Corporation 26
Containers and container orchestration overview!
• Docker is an open technology platform for building and shipping applications as self-contained container images, portable across any Docker runtime
• Containers provide an operating system environment to applications, but use core operating system capabilities of the host system – containers run as sandboxed processes within the host OS
• Container images can be customized towards the specific needs of each application
• Container-based applications (e.g. in micro-services architectures) typically consist of multiple containers with inter-dependencies
Build Ship Run
https://www.docker.com
container dependencies scaling dependencies on
other resources
© 2015 IBM Corporation 27
docker-compose!
• A tool for defining and running multi-container applications with Docker
• Multi-container pattern defined in a docker-compose YAML configuration file
• Ability to create, start, stop, delete complete multi-container deployment
• Ability to operate containers individually
• Ability to scale containers defined in the template
https://docs.docker.com/compose/
db: image: db_image:1.0 environment: some_var: some_value volumes: -‐ /var/my_files:/opt/my_db/data app: image: app_image:2.0 ports: -‐ "80:8080" links: -‐ db:database
link to other service "db" using alias "database"
externally exposed ports
© 2015 IBM Corporation 28
Kubernetes!
• Open-source platform for automating deployment, scaling, and operations of container-based application across clusters of hosts
• Initiated by Google and donated to Cloud Native Computing Foundation in July 2015
• Definition of Pods consisting of one or more containers • Pod is a definition of a desired
state (all containers running) that is ensured by the kubelet engine
• Automatic placement and scaling of Pods across nodes ("minions") in a cluster
http://kubernetes.io
apiVersion: v1 kind: Pod metadata: name: www spec: containers: -‐ name: nginx image: nginx volumeMounts: -‐ mountPath: /srv/www name: www-‐data readOnly: true -‐ name: git-‐monitor image: kubernetes/git-‐monitor env: -‐ name: GIT_REPO value: http://github.com/some/repo.git volumeMounts: -‐ mountPath: /data name: www-‐data volumes: -‐ name: www-‐data emptyDir: {}
Pod definition
© 2015 IBM Corporation 29
Agenda!
• Introduction
• Pattern Modeling Concepts
• OpenStack Heat
• Container Orchestration
• Hybrid Orchestration
© 2015 IBM Corporation 30
Orchestration of hybrid workloads!
app app
middle-ware
middle-ware
VM VM VM
Docker
app app *aaS
containerized components full-stack deployments as-a-service components
VM VM
MW MW
MW MW
MW container
Service
© 2015 IBM Corporation 31
Hybrid pattern orchestration in OpenStack Heat!
• In case of mixed deployments (containers, full stack, external services) a generic orchestration engine with pluggable support for different target platforms and technologies is required
• Heat allows for using resource type plug-ins (see Heat resource model) for orchestrating any API-based resources • Out-of-the-box support for OpenStack Magnum resources to manage
containers through the Magnum container service as part of a Heat stack
• Resource type plug-ins can be implemented for any *aaS service to be included in a hybrid Heat stack
© 2015 IBM Corporation 32
Summary!
• Orchestration describes the automated arrangement, coordination, and management of complex computer systems, middleware, and services • Orchestration can be done based on patterns as structural models of application-
and infrastructure components and relationships between those components • Patterns are processed (primarily) in a declarative way, deriving orchestration flows
from the structural model of patterns
• OASIS TOSCA describes pattern elements and concepts in a standardized way
• OpenStack Heat is a pattern-based orchestration engine on-top of base OpenStack Service, and has evolved as the de-facto open pattern technology
• Container technologies (e.g. Docker) are rapidly gaining traction for deploying and managing micro services based applications • Orchestration technologies for Docker containers have evolved that are based on
simple multi-container environment patterns
• Orchestration of hybrid environments (e.g. full stack + container + *aaS) requires use of generic orchestration technology (e.g. Heat) able to cover the superset of pattern components