Workflows as Commodity: Key Issues for the Future · 1 Prof. Dr. Frank Leymann Institute of...

1

Prof. Dr. Frank LeymannInstitute of Architecture of Application Systems

(IAAS)

University of StuttgartUniversitätsstr. 3870569 StuttgartGermany

Phone +49-711-7816 470Fax +49-711-7816 472 e-mail Frank.Leymann

@informatik.uni-stuttgart.de

Workflows as Commodity:Key Issues for the Future

( Nancy, France, September 2005 )

2© Frank Leymann

2

3© Frank Leymann

Agenda

Web Services as CommodityApplication StructuresUtility ComputingFlow OptimizationSummary

4© Frank Leymann

Agenda


3

5© Frank Leymann

EAI – The Problem

SAP Order Entry

SiebelUpdate

Customer Record

OracleUpdate Available

Credit

CICSStockHold_Tran

DB2Log

High-Value Order

ERPERP

CRM TPM

DBS

6© Frank Leymann

Application Bridges

Application A

אטנצשה

Application B

A B Bridge

CICS EXCI,SAP RFC,OS CALL,

...

4

7© Frank Leymann

אטנצשה

Adapters And Queuing: Advent of QoS

Application A

אטנצשה

Application B

SourceAdapter

TargetAdapter

Source Environment

Target Environment

MessageQueuing

8© Frank Leymann

Advent of Brokers

MessageBroker

Message xform,Side effects,

…(“Side Effects”)

Message xform,Side effects,

…(“Side Effects”)

5

9© Frank Leymann

Flows For “Side Effects”

GET msgFROM queue

XFORM into adapter_1 format

OrderValue > 10k€

STORE msg into DB

PUT msg INTO Q1(target adapter_1)

Format unknown

APPEND to ErrorLog

SEND eMailto Admin

XFORM into DB format

10© Frank Leymann

Queuing-Based Integration Architecture

Message Queuing System

Broker µFlowEngine

Analyze msg & call corresponding µflow

Application i Application j Application k

אטנצשה

ЖЦДБ

אטנצשה

(Source)Adapter

(Target)Adapter

(Target)Adapter

6

11© Frank Leymann

Application Server-Based Connector Architecture

AppServer Core

Extensions ForConnector Plugability

Application specificConnector

ApplicationServer

Target Application

µFlowEngine

JMS

Source Application

asdfad

אטנצשה

Source A

pplication

12© Frank Leymann

Wrapper: Some Sort Of Adapters

DBMS DBMS Extensions for

wrapper plugability

SQLSQL

Wrapper Wrapper Wrapper Wrapper

OtherDBMS

StandardApplication

FileSystem

LegacyApplication

SELECT...FROM...WHERE

7

13© Frank Leymann

Flows & Wrappers In Databases

DBMS

SQLSQL

WFMS

SELECT...FROM …,T,… WHERE...

TWrapper

ERP PDM HR RYO…

14© Frank Leymann

Flows & Channels In B2B Interactions

Enterprise B

PublicProcess

Enterprise A

PublicProcess

Channel

Channel

8

15© Frank Leymann

And So On?

Well, this integration technology is confusing:AdaptersConnectorsWrappersChannels…

Where does it stop?

Web Services !!!

16© Frank Leymann

Web Service Technology: The Key Thing!

1. Web service technology provides a “virtual component model“ for using components in a loosely coupled manner

2. When using a Web service the supporting container hides its “middleware idiosyncrasies“ (component model behind the implementation of the Web service, the invocation protocol etc.)

3. Web service technology does not provide a new model for implementing components(with the exception of BPEL ☺)

1. Web service technology provides a “virtual component model“ for using components in a loosely coupled manner

2. When using a Web service the supporting container hides its “middleware idiosyncrasies“ (component model behind the implementation of the Web service, the invocation protocol etc.)

3. Web service technology does not provide a new model for implementing components(with the exception of BPEL ☺)

9

17© Frank Leymann

DBMS .NETJ2EE

Virtual Component

Concrete Component

Virtualizing Components

AssemblyStP ...

WebService

(E)JB

implementsimplementsimplements

18© Frank Leymann

The Ordering and Guiding Principles

All standard efforts should foster…1. an environment which is

a. heterogeneous, b. distributed, c. interoperable,d. loosely coupled,e. multi-vendor

2. secure, reliable, recoverable interactions3. a single uniform usage model for “components”4. business criteria to discover components needed

“The Service Bus”!“The Service Bus”!

10

19© Frank Leymann

Organizing Into A Platform

Messaging

Quality of Service

Transport

Description

Transports

Interface + Bindings

Composite

XML Non-XML

Security

Policy

Discovery, N

egotiation, Agreem

ent

Atomic

ChoreographyChoreography ProtocolsProtocols StateState Components

ReliableMessaging Transactions

20© Frank Leymann

TransactionsReliableMessaging

The Bus And Standards

Transports

Interface + Bindings

Composite

XML Non-XML

Security

Policy

Discovery, N

egotiation, Agreem

ent

AtomicChoreographyChoreography GroupingGroupingBPEL

WS-RM WS-Security* WS-AT,WS-BA,…

WSDL* WS-Policy*

SOAP, WS-Addressing JMS, RMI/IIOP, ..

HTTP, TCP/IP, SMTP, FTP, …

UD

DI, W

S-A

ddressing, ME

X,…

WS-C,WS-N*,… StateStateWS-RF

Messaging

Quality of Service

Transport

Description

Components

11

21© Frank Leymann

cÉÄ|vç +

Matching Endpoints Based On Policies

R

Candidate ListSelected Port

Service Discovery

?

!

22© Frank Leymann

Virtual Service

Service Bus

Business Properties

Don‘t Care! Virtualizing Services

I want ...But I don‘t care about a particular

provider: Chose the one who is at this point in time „the best“ for me!

EnvironmentalProperties

cÉÄ|vç

12

23© Frank Leymann

Distributed Heterogeneous Environment

Distributed physical clusters and storage

24© Frank Leymann

Virtual clusters and storage

The Grid: Virtualizing Resources

Grid MiddlewareService Bus !

OGSAOGSA

13

25© Frank Leymann

Agenda


26© Frank Leymann

Reminder: Process-Based Applications

Business Process Models

Business Functions

….EXE CICS MQ EJB (D)COM+

14

27© Frank Leymann

Modeling Business Processes

What?

Who?

With?

GetOrder

CheckCustomer

AcceptOrder

RejectOrder

Always!Bad Customer?

Good Customer?

Credit CardInformation

CustomerAddress

28© Frank Leymann

Microflows Are Business Processes

GetOrder

CheckCustomer

AcceptOrder

RejectOrder

Always!Bad Customer?

Good Customer?


CustomerAddress

What?

Who?

With?

Who?

15

29© Frank Leymann

Business Processes Between Web Services

X

Y

Z

pt3

o1

o2

pt1

o1

o3

o4

pt2

o2

o4

30© Frank Leymann

Business Processes As Web Services

X

Y

Z

pt

o1

o2

pt*o7

o8

o9

16

31© Frank Leymann

Summary: Aggregating Web Services

X

Y

Zpt

o1

o2

pt3

o1

o2

pt1

o1

o3

o4

pt2

o2

o4

P

Q

pt*o8

o9

BPEL4WSBPEL4WS

32© Frank Leymann

The Notion Of “Orchestration”

Business Process Models

Business Functions

….EXE CICS MQ EJB (D)COM+

Web Services

Orchestration

17

33© Frank Leymann

Programming in the Small

Programming in the Large

Programming Model

Workflow System

Application Server

Choreography/Flow/Process

Web Services/FunctionsApplication

34© Frank Leymann

Runtime: The Need For Deployment

A

C

B

D

E

Process

Bus

Application

DeploymentDescriptors

DeploymentDescriptorsActivity:Port TypeOperationBizPropertiesQoS... cÉÄ|vç

Ports

18

35© Frank Leymann

Programming Model: Refinement

Flow

Components

Application

WorkflowSystem

ApplicationServer

Programmingin theLarge

Programmingin theSmall

Deployment ServiceBus

DeploymentDescpT pLink Locator

36© Frank Leymann

Supporting People Activities

X

Y

Z

pt1

o1

o3

o4

☺(abstract organizational group)

BPEL4PeopleBPEL4People

19

37© Frank Leymann

Deployment Descriptor

…

Supporting People: Deployment

A

ApplicationArtifacts

staff query…

☺

Org Database

Task

u

38© Frank Leymann

Workflow Language Genealogy (incomplete)

FDL

FDML

XPDL

BPXL

WSFL

BPEL

XLANG

BPML WSCI BPSS

WS-CDL

(IBM)

(IBM)

(IBM)

(Microsoft)

(WfMC) (bpmi.org) (SUN, SAP,Oracle)

(OASISebXML)

(bpmi.org)

(W3C)

(IBM, Microsoft, SAP, BEA OASIS)

BPELJ

YAWL GSFL…

(Research)

?…?BPEL4People

20

39© Frank Leymann

O

O’

M

The amount of the order must at least

be 100€

A

B

C

Business RepresentationBusiness Representation

assign send

≤ xcM: …

IT RepresentationIT Representation

Model Representations and Semantics

Process model often has different representation at business level and IT levelSemantic descriptions help to identify/match them

40© Frank Leymann

A

B

C

O1 O2 O3

Deriving Services From Semantics

Selection of appropriate port types may be done base on semantic descriptions at the business level

Ô1 Ô2 Ô3

SemanticWeb Services

21

41© Frank Leymann

≤ xcM: …

O1 O2 O3

O

Deriving Semantics of Process Models

Semantic description of (fragments of) process model may be derived from semantic Web services and (causal,… relations expressed via) process model itself

42© Frank Leymann

…

…

…

≤ xcM: …

Business IT Transformation

Fragments of process models may be translated into IT variants based on semantic matchmaking

O

22

43© Frank Leymann

Solutions: Packaging Application Artifacts

Choreography

Solution

<other artifacts>Services!

Rules …

A solution is a package containing all the software artifacts required to solve a business problemA solution does not assume additional programming in order to become executableBut some of the artifacts of a solution might require binding/deployment such that the solution becomes executable

44© Frank Leymann

Application Template

…is a package that enables customization…might even require implementation work to result into a solution

For example, process model may only be partially defined, message structures may be only course grained, rules may only be sketched,…BPEL “opaque”

…may represent “best practices” that have to be refined at a particular site

Points-of-variability specify what can be changed without jeopardizing integrity or that must be completed/refined

23

45© Frank Leymann

What Is Typically Customized

TravelServiceorderTrip

receiveItinarary

getHotelgetFlight

chargeCC

HotelServicebookRoom

AirlineServicebookFlight

CCServicepayBill

X<5

“Points-of-Variability: Application Logic”

“Points-of-Variability: Environment”

46© Frank Leymann

Environment Customization Tool(SLAs, Deployment Descriptors,…)

Customizing Application Environment

pT1

pT2

pT3

pT4

pT5

pT6

pT7

pT8

TravelServiceorderTrip

receiveItinarary

getHotelgetFlight

chargeCC

HotelServicebookRoom

AirlineServicebookFlight

CCServicepayBill

X<5

€X>13

EnvironmentElement

EnvironmentElement

24

47© Frank Leymann

Finally: Solution And Environment

Business Process

Components

HardwareDBMSOperating System

Middleware

48© Frank Leymann

Even Then: Runtime Adaptability Needed

What?

Who?

With?

Logic adaptation(Ad hoc, instance migration,…)

GetOrder

CheckCustomer

AcceptOrder

RejectOrder

Bad Customer?

Good Customer?


CustomerAddress

Always!

Binding adaptation(Deployment, late binding, semantics,…)

Organizational adaptation(Staff assignment, abstract roles,…)

Environment Adaptation

25

49© Frank Leymann

Agenda


50© Frank Leymann

ManageableResource

Manageable Resource

ManageabilityInterface

Allows to plug resource into a management system.It makes the resource manageable

(e.g. lifecycle, monitor, metadata, state…)

Allows to plug resource into a management system.It makes the resource manageable

(e.g. lifecycle, monitor, metadata, state…)

ResourceInterface

Resource specific functions, e.g.mountDisk, openDatabase, allocateCPU,…

Resource specific functions, e.g.mountDisk, openDatabase, allocateCPU,…

26

51© Frank Leymann

Effectors

Sensors

Manageability: Sensors And Effectors


ResourceInterface

Effectors enable explicit state management by providing lifecycle operations

and update operations on state and data

Effectors enable explicit state management by providing lifecycle operations

and update operations on state and data

Update Properties

Sensors enable monitoring by providing queries (“pull”)on properties (actual state, metadata and data)

as well as change notification (“push”)

Sensors enable monitoring by providing queries (“pull”)on properties (actual state, metadata and data)

as well as change notification (“push”)

Get Properties

52© Frank Leymann

Resources Are Web Services ☺

ManageableResource


ResourceInterface

Properties(status,data,

metadata)

Port Type

Concept SpecificationWS-Resource FrameworkWS-Resource Framework

27

53© Frank Leymann

Notification: Base For MonitoringWS-Notification*WS-Notification*

SubscriberNotification

SourceNotification

Sink Y

Bus

Broker

Subscribe sink Y to events X

Publish notification about X

Deliver notification message

54© Frank Leymann

Service Bus

MAPE and Service Bus

Sensors Effectors

Resource

Execute

PlanAnalyze

Monitor

1

2

3

4

5

6

InvocationFeatures

Publish/SubscribeFeatures

Monitor

Analyze Plan Execute

28

55© Frank Leymann

Service Bus

Management: Using The Bus

... ...

Monitor Operate Configure...

WSDMWSDM

56© Frank Leymann

HardwareDBMS

OperatingSystem

Middleware

Components

Business Process

Computing On Demand

On

Dem

and

Infr

astru

ctur

e

In 3 hours I need the following environment for 45 minutes: 253 machines, 1.7 TB storage,

WebSphere, DB2, and the follow ERP applications…

cÉÄ|vç SLASLA

29

57© Frank Leymann

The Ingredients Of On Demand

On Demand

Web Services

BusinessProcesses Grid

Autonomic

58© Frank Leymann

Outsourcing(Information Technology)

30

59© Frank Leymann

Terminology

OutsourcingAllocation of IT orders to external companies (e.g. running of [parts of] IT infrastructure including application functionality, development of application functions)

Off-ShoringSpecial case of outsourcing: external company is in foreign country

60© Frank Leymann

Situation

ASP* model was never really successful:Model could not exploit economy of scaleInteroperability problems

Technology was not ready for this!Recent advancements in Web service-, Grid-, autonomic-, and workflow-technologies are about to remove these obstructions

Recently, outsourcing of both, application functions and business processes is rising sharplyNumber of intermediate aggregators is rising

Shift from ASP model to Utility model!Shift from ASP model to Utility model!

(*) Application Service Provider

31

61© Frank Leymann

The Change

Providers become more cost-effective based on being able to exploit economies of scale

Just like traditional public utilities do since decadesThe key enabler is “dynamic provisioning”

I.e. making resources available only when needed“Resources” = hardware, storage, networking, middleware, application functions, business processes

Consumers get more trustThey only have to pay what they useDependency on third-party – i.e. provider lock-in – is significantly reduced

Web service interoperability allows to change providers without major re-write of software

Message architecture, security, interfaces, policies, processes, SLAs,…

62© Frank Leymann

Reasons For Outsourcing

They are manifoldThey and their priority change(d) over timeCost-savings is mostly ranked high

Maintenance of infrastructure and softwarePurchase cost often not so criticalKeeping it current is expensive

Skilled IT personnel is rareOperators, systems management personnelComplexity of IT is constantly raising Training (costly) to avoid (costly) outages,…Programmers

Focus on core competencies is importantIT is not the proper business of companies

Quality improvement is importantApplication backlog is obstruction to proper functionalitySkills of own personnel in some areas not appropriate

34

67© Frank Leymann

Outsourcing Highly-Customized Applications

68© Frank Leymann

A Pessimistic Scenario

Your headquarter “Your” compute center(business processes)

“Your” compute center(application functions) “Your” programmers

35

69© Frank Leymann

Geographic Distribution of Application

Match/Bind

Your and your application

70© Frank Leymann

But There Is Hope: Shift of Skills

Your headquarter “Your” programmers“Your” Operators

Your IT architects

Your process specialists

36

71© Frank Leymann

And There Are Counter-Arguments !!!“Off-shoring of infrastructure is high risk”

Low-income countries sometimes in political, geographical,… risky areas of the worldI.e. higher probability of outages of infrastructure

“Outsourcing of programming is brain-drain”Company is loosing important skillsYou become dependent on third party

“Salaries in low-income countries are raising”Outsourcing/off-shoring is only temporary; try to avoid it

“Outsourcing/off-shoring of programming has hidden costs”Functional requirements must be documented down to very low level Lot’s of meetings, reviews,…

Increased costs for remaining staffQuality is not up to standards used too

“Legal aspects are painful”SLAs negotiation is very criticalAre electronically negotiated contracts legally binding?

Automatic, dynamic select of providers assume corresponding contracting & negotiations

Cross-country contracts (off-shoring) produce additional costs requiring special lawyer skills

…

72© Frank Leymann

Pragmatics

Reality will be somewhere in between ☺Reality will be somewhere in between ☺

38

75© Frank Leymann

Today…

Everybody wants to go with the flow!Everybody wants to go with the flow!

76© Frank Leymann

We Saw Just a Few Minutes Ago…

…the use of flow technology in:Enterprise application integrationDatabase technologyProvisioningPackaged application structures

…and don’t forget the “classical” usage of workflow technology

39

77© Frank Leymann

…And There Is Much More “Demand”!!!

Portal personalizationWarehousingAdaptive ApplicationsPeer-To-Peer FlowsGrid applicationsGrid Transaction-Management …

78© Frank Leymann

Thus, Soon…

Nobody can go without flow!Nobody can go without flow!

40

79© Frank Leymann

A Déjà Vu – Nearly ☺

Panta Rei (l Everything flows)(Heraklit, 550 B.C. - ~480 B.C.)

Panta Rei (l Everything flows)(Heraklit, 550 B.C. - ~480 B.C.)

80© Frank Leymann

Consequence

Flow technology becomes extremely critical in terms of performance (= overall runtime overhead)!!!

WFMS must become “homeopathic” ☺

Remember: Availability and scalability already addressed by today’s production workflow systems

41

81© Frank Leymann

A Unifying Concept: Behavior Spheres

µFlow

ACIDic

Critical Response Time Same Performer

Data Centric

Compensatable

Non-Executable

82© Frank Leymann

The Conclusion

Flow technology is now where RDBMS technology was about 20 years ago:

Initial success resulted in huge demand from broad spectrum of application areas

Different application areas require different optimized behavior This is what we have to focus on too:

Flow Optimization!

42

83© Frank Leymann

Flow Optimization: Next Steps

As in RDBMS this has two major aspects

Algebraic optimizationSyntactic rewrite of request – in flow speech:

Graph surgery based on Behavior Spheres

Heuristic optimizationStatistics and cost based plan derivation – in flow speech:

Audit trail based graph rewrite using analysis and mining technologies

84© Frank Leymann

And This Is A Long-Lasting Vision?

Yes, it is!

Flow technology has matured, it becomes mainstream – finally

The optimization vision is long-lastingWill take a decade of research & development!Complex technology, challenging jobs!

It will be “blood, sweat, and tears” !!!

43

85© Frank Leymann

Agenda


86© Frank Leymann

Conclusion

Web service technology and workflow technology is about to become a commodityService Bus & WfMS are underlying middleware Process Models as base of new tradable artifacts“Optimization” must become focus of workflow research and developmentAutonomic and On Demand based on the aboveSocio-political impact

Programming-in-the-small in “low-income” countriesOutsourcing, offshoring

Programming-in-the-large “anywhere”Especially in “high-income” countries & close to using companies

Impact on (university) education

Workflows as Commodity: Key Issues for the Future · 1 Prof. Dr. Frank Leymann Institute of...

Documents

Transcript of Workflows as Commodity: Key Issues for the Future · 1 Prof. Dr. Frank Leymann Institute of...