From ITIL to DevOps Orchestration & Choreography in Cloud ... · PDF file© 2012 IBM...

© 2012 IBM Corporation

Simon Moser, IBM

Cloud Computing Architect

- From ITIL to DevOps : Orchestration & Choreography

in Cloud Services and Environments -


Agenda

1

Self-Service Orchestration Models

2 Lifecycle Phases of an IT Service and their Orchestration

Introduction to Orchestration & Choreography in the Cloud

3

1


Definition: What is Choreography, What is Orchestration ?

These terms have an intentional connotation with music:

choreography represents a set of services that work together

to achieve a larger goal; however, each service acts in an

individual way – similar to dancers in a ballet. On the contrary,

in an orchestration, a set of services are orchestrated by a

“conductor”, i.e. a main service that orchestrates, or

“conducts”, all participating services.

J.Cardoso and W. van der Aalst, editors, Handbook of Research on Business Process Modeling


Terminology: What is a „Service Orchestration“ ?

For those with a background in SOA: The terms „Service“ and „Orchestration“ are used with

a different focus in SOA and Systems Management

“Service” means … – In SOA: Any kind of (reasonably coarse-grained) application function

– In Systems Management: Any kind of resource and appropriate actions required to

support business with IT

“Orchestration” means …

– In SOA: the aggregation of application functions into higher level business functions

– In Systems Management: the proper sequencing of individual management tasks to

manage complex IT artifacts

– In the Cloud: Describes „Processes“ (both business processes and workflows) needed to

achieve „things“ in Systems Management

Both notions of “orchestration” can be realized with the same underlying technology (BPMN,

BPEL – we will get to that …), but the focus is very different

© 2012 IBM Corporation 5

Infrastructure-as-a-Service

Platform-as-a-Service

Application-as-a-Service

Servers Networking Storage

Middleware

Collaboration

Financials

CRM/ERP/HR

Industry

Applications

Data Center

Fabric

Shared virtualized, dynamic provisioning

Database

Web 2.0 Application

Runtime

Java

Runtime

Development

Tooling

Cloud Computing Layers & Service Types

Business Process-as-a-Service

Employee

Benefits Mgmt.

Industry-specific

Processes

Procurement

Business Travel


Service Runtime /

Management Phase

Service Instantiation Phase

Service Lifecycle Phases and their orchestration

Service Refinement Phase Service Definition Phase

Service Destruction Phase (Infrastructure) Service

Construction / Integration

Phase

3 4

2 5

1 6

Phase 1: (Infrastructure) Service

Integration / Construction

Orchestration Use Case: New Storage Service

8


Introduction to IT Infrastructure Library (ITIL)

Founded in the 80s

Collection of “Best Practices“ or “Good Practices“

– Version 1 1989 – Version 2 2000 – Version 3 2007

Framework of processes to manage IT as Service

Really a common language with well-defined terms and procedures

Becoming more and more relevant

“Implementations“ from different vendors available for different industries

– Microsoft, Oracle, HP, IBM


The Notion of a Process according ITIL

Characteristics of a process – produces a defined output – processes are measurable – across organisation and functions – continious improvement of output

Definition:

A process is a structured set of activities to produce a well defined output

Process

outputs

Roles within Process

– Set of responsibilities

assigned to a person or group

– A person or team can be part

of one or more roles

Process

inputs

Process Control

Process Owner Process Objectives

Process Enablers

Process Resources Process Capabilities

Process

Activities Metrics Roles Procedutes

Instructions Improvements

Quelle: OGC Service Strategy


Orchestrations used in (Infrastructure) Service Integration Phase

ITIL Processes

Change Management - Minimize the number and impact of incidents related to planned changes to control the IT Infrastructure

Service Catalog Management

Other Processes or workflows

None

Phase 2: Service Definition

13

Orchestration Use Case: Automate VM Creation

Service Providers in a Cloud Infrastructure

Orchestration Use Case: Automate Application Onboarding


Recap on TOSCA: Topology and Orchestration Specification for Cloud Applications


Sugar CRM Topology Template

VM-Apache

(VirtualMachine)

OS-Apache

(OperatingSystem)

ApacheWebServer

(ApacheWebServe

r)

SugarCRM App

(SugarCrmApp)

PHP Module

(PhpModule)

HostedOn HostedOn

HostedOn

HostedOn

DependsOn

VM-MySQL

(VirtualMachine)

OS-MySQL

(OperatingSystem)

MySQL

(MySQL)

SugarCRM DB

(SugarCrmDb)

HostedOn

HostedOn

HostedOn

ConnectsTo


Sugar CRM Deployment Artifacts

VM-Apache

(VirtualMachine)

OS-Apache

(OperatingSystem)

ApacheWebServer

(ApacheWebServe

r)

SugarCRM App

(SugarCrmApp)

PHP Module

(PhpModule)

HostedOn HostedOn

HostedOn

HostedOn

DependsOn

VM-MySQL

(VirtualMachine)

OS-MySQL

(OperatingSystem)

MySQL

(MySQL)

SugarCRM DB

(SugarCrmDb)

HostedOn

HostedOn

HostedOn

ConnectsTo

1 5 2

3 4

6 6


Orchestrations used in Service Definition Phase

ITIL Processes

Service Request Management - Process that describes Service Onboarding into the Cloud


Whatever the Application model defines, e.g.

Application Build Plan

Application Termination Plan

Application-specific Backup Plan

Application-specific Patching Plan

...

Phase 3: Service Instantiation

Orchestration Use Case: Application Build Plan


Recap: Workflows, BPMN, BPEL, ...

Workflow = Orchestrating a set of a tasks according to a predefined „template“

Process Template / Plan = Defines how an “incident” must be handled. A process is usually represented as a graph. Nodes are “activities” or “tasks”, edges define the control- and/or data flow

Activity / Task = Basic unit of a process. Activities can contain other activities, similar to blocks in regular programming languages.

Service = An Activity / Task can be implemented through a Service, e.g. a Program, Application or Script

Common languages to describe workflows: – BPMN – BPEL


What is BPEL?

Business Process Execution Language, OASIS Standard since April 2007

High-level programming language

Recursive, service-based programming model with defined execution semantics

WS-BPEL hat build-in support for

Long-running Processes

Scalability

Parallelism

Message- and Instance Correlation

Error handling

Compensation- based recovery

Phoenix - behavior


WS-BPEL: Basic Activities

Do a blocking wait for a matching

message to arrive

Send a message in reply to a

formerly received message

Invoke a one-way or request-

response operation

Update the values of variables or

partner links with new data

Validate XML data stored in

variables

No-op instruction for

a business process

Generate a fault from inside the business process

Forward a fault from inside a fault handler

Immediately terminate execution of a business

process instance

Wait for a given time period or until a certain time

has passed

Invoke compensation on an inner scope that has

already completed

Wrapper for language extensions

receive

reply

invoke

throw

exit

wait

empty

compensate validate

assign

rethrow

extensionActivity


WS-BPEL: Structured Activities

Contained activities are executed in

parallel, partially ordered through

control links

Contained activities are performed

sequentially in lexical order

Contained activity is repeated while a

predicate holds

Contained activity is repeated until a

predicate holds

Block and wait for a suitable message to

arrive (or time out)

Contained activity is performed sequentially

or in parallel, controlled by a specified

counter variable

Select exactly one branch of activity from a

set of choices

Associate contained activity with its own

local variables, fault handlers, compensation

handler, and event handlers

if then else while

scope

pick

sequence

flow

repeatUntil

forEach


What is BPMN ?

BPMN 1.x

– Business Process Modeling Notation

– Current version 1.2 (January 2009)

• http://www.omg.org/spec/BPMN/1.2/

BPMN 2.0

– Business Process Model and Notation

– Standard since 1/2011

– Process Modeling Conformance

• Support modeling of processes and collaborations

• Corresponds roughly to the BPMN 1.1 set of capabilities

– Choreography Modeling Conformance

• Support modeling of choreographies

– Process Execution Conformance

• Support execution of BPMN processes according to defined execution

semantics

– BPEL Process Execution Conformance

• Support execution of BPMN processes by mapping to BPEL as per the defined

BPMN-BPEL mapping.

http://www.omg.org/spec/BPMN/1.2/


Overall Structure of a BPMN Process

Lanes

Activities

– Structured

– Atomic Tasks

Events

– Inbound

– Outbound

Ordering

– Sequence Flow

– Gateways

– “Adornments”

Pro

ce

ss O

rgU

nit2

Org

Un

it1


Orchestrations used in Service Instantiation Phase

ITIL Processes

Request Fulfilment

– Information, advice or a standard change that should not be classed as Incidents or Changes

Access Management

– Right things for right users at right time – Concepts:

• Access • Identity (Authentication) • Rights (Authorisation) • Service Group • Directory


Whatever the Application brings with it … ´

Question: How can a Service that does specific orchestration like instantiation integrate with

the larger operational processes (ITIL) that his hosting cloud might require ?

Phase 4: Service Runtime (Management) Phase

Orchestration Use Case: Broken Hardware in a Data Center

What happens in a data center if a hard drive crashes ?

1. Operator gets notified (a ticket) that a hardware outage occured 2. Operator walks over to the affected machine and checks what was broken 3. Operator checks if he has a replacement available 4. If no – Operator orders a replacement 5. Operator replaces broken disk with replacement disk 6. Operator closes the ticket

Orchestration Use Case: Broken Hardware in a Cloud

What happens in a cloud if a hard drive crashes ?

1. Operator gets notified (a ticket) that a hardware outage occured 2. Operator / Automated Plan deactivates the broken hard drive, swapping in other

storage 3. At scheduled maintenance time, operator replaces broken disk with replacement disk 4. Operator closes the ticket

Orchestration Use Case: Application specific Backup


Orchestrations used in Service Runtime Phase

ITIL Processes

Incident Management - Deals with unplanned interruptions to IT Services or reductions in their quality

Event Management – 3 Types of events: Information, Warning, Exception

Change Management – Minimize the number and impact of incidents related to planned changes to control the IT

Infrastructure

Problem Management – Aims to prevent problems and resulting incidents – Minimizes impact of unavoidable incidents – Eliminates recurring incidents


Whatever the Application brings along (e.g. Application specific backup)

Phase 5: Service Refinement Phase

Development & Operations (DevOps)

• “A methodology to bridge the traditional separate departments for development, IT operations and quality assurance and to combine the different aspects of development and deployment activities to provide an increased rate of production releases from application and business unit stakeholders, realized through a high degree of automation on virtualized infrastructure.”

• Continuous Delivery -> realized through scripts

Orchestration Use Case: Continuous Delivery

36

Orchestration Use Case: Application

Placement Plans Cheap

Storage

Phase 6: Service Deletion Phase

Orchestration Use Case: Service Termination Phase


Orchestrations used in Service Termination Phase

ITIL Processes

(Termination) Request Fulfilment

– Information, advice or a standard change that should not be classed as Incidents or Changes

Access Management

– Right things for right users at right time – in this case remove


Whatever the Application brings with it … ´

Question: How can a Service that does specific orchestration like instantiation integrate with

the larger operational processes (ITIL) that his hosting cloud might require ?

SaaS and BPaaS related Orchestrations


Self-Service Orchestration

Summary

A Business Process that is exposed as a cloud

service

Something that I define out of existing building

blocks

Product / Offering

Cloud Portal + APIs

Roles

Cloud Service User

Example

Travel Approval

Video Encoding

© 2012 IBM Corporation 43

Infrastructure-as-a-Service

Platform-as-a-Service

Application-as-a-Service

Servers Networking Storage

Middleware

Collaboration

Financials

CRM/ERP/HR

Industry

Applications

Data Center

Fabric

Shared virtualized, dynamic provisioning

Database

Web 2.0 Application

Runtime

Java

Runtime

Development

Tooling

Recap: Cloud Computing Layers

Examples

Business Process-as-a-Service

Employee

Benefits Mgmt.

Industry-specific

Processes

Procurement

Business Travel


Software as a Service or Business Process as a Service

Whats the difference between Application as a Service and BPaaS

– AaaS = Running code as a service

– BpaaS = Involvement of humans

Which role defines the „workflow model“ ?

– BPaaS = Cloud Service Provider defines the model

Are there other options than that ?


Amazon SWF = Process Engine as a Service

So the process model is in the “Decider”

But the workflow is code - not a model!

Workflow SDK as a cloud application -

Engine, Scheduler, Auditing, etc.

WebService–Interface, invokable by many programming languages

You register activity types (“Workers”)

A “Decider” orchestrates over the “Workers”


Process Engine as a Service : Why ?

Because you can define the higher-value service myself

Because you are not restricted to what my favorite cloud provider offers me

Scalable, resilient, distributed applications

Run application workflows and business processes in the cloud

Reliable, auditable, periodic execution

Asynchronous- and Event Driven Execution


Conclusion & Challenges

ITIL processes etc. are well understood and well established

Service models bring along service specific orchestration, independent of where the Service will be hosted

Cloud management software knows how to execute such plans, but misses integration with the larger

operational processes (ITIL)

==> Challenge: How can we make integration between ITIL process and service-specific plans more

seamless ? What additional capabilities would a Cloud Management software have to provide ?

Pre-canned service blocks (both AaaS and BPaaS) come with orchestration scripts fit for purpose

Self-service orchestration comes at a higher self cost but also with higher flexibilitys:

• You need to understand what „building blocks“ are available

• Not „ready to use“ - you need to write the „orchestration script“ yourself

==> Challenges: Where is the difference about orchestration scripts as code (Amazon SWS, CHEF) vs.

orchestration scripts as process models (TOSCA, BPMN)? Do certain roles have Preferences ? If so –

why ? Are there advantages from one over the other ? Does this evaluation change if you switch the role

from e.g. a cloud service provider to a cloud user ?


THANK YOU

From ITIL to DevOps Orchestration & Choreography in Cloud ... · PDF file© 2012 IBM...

Documents

Transcript of From ITIL to DevOps Orchestration & Choreography in Cloud ... · PDF file© 2012 IBM...