1ICTEMMSAD’05 13/6-2005

Assessing Business Process Modeling Languages Using a Generic Quality Framework

Anna Gunhild Nysetvold*

John Krogstie *,§

IDI, NTNU * and SINTEF§

Norway

2ICTEMMSAD’05 13/6-2005

Overview of presentation

Quality of models and modeling languages Objectives of business process models Description of case - Vital Evaluation results Concluding remarks

3ICTEMMSAD’05 13/6-2005

Framework for quality of models

E m pirica lqua lity

S ocia lqua lity

P hysica lqua lity

S ocia lp ragm atic

qua lity

S em anticqua lity

S yntacticqua lity

Techn ica lp ragm atic

qua lity

M ode lexterna liza tion

M

Socia lactor

interpretationI

Technicalactor

interpretationT

M odelingdom ainD

Languageextension

L

M odellerexplic it

knowledgeKM

P erce ivedsem antic

qua lity

Socia l actorexplic it

knowledgeKS

G oals ofm odelling

G

O rgan iza tiona lqua lity

4ICTEMMSAD’05 13/6-2005

Quality of modeling languages

Know ledge externalizab ilityappropria teness

Technical actorin terpretation

appropria teness

Com prehensib ilityappropria teness

M odelexterna liza tion

M

Socia lac tor

in terpre ta tionI

Techn ica lac tor

in terpre ta tionT

M odelingdom ainD

Languageextension

L

M odelle rexp lic it

know ledgeKm

D om ain appropriateness

M odelling G oa lG

Soc ia l ac torexp lic it

know ledgeKs

Partic ipant languageknow ledge appropriateness

O rgan isationalappropria teness

5ICTEMMSAD’05 13/6-2005

Different objectives of business process modeling

Human-sense making and communication Computer-assisted analysis/simulation Business Process Management Gives the context for a traditional system development project Model deployment and activation :

Through people guided by process 'maps', Automatically, as in most workflow engines. Interactively, where the computer and the users co-operate

6ICTEMMSAD’05 13/6-2005

Description of case-study environment Vital: One of Norway’s largest insurance companies Large number of life insurance and pension insurance

customers Going from a functionally oriented architecture to a

process/service oriented architecture Need to support complete business processes in the

architecture Main usage area of process models: Context for system

development (but human sense-making and communication is also important)

7ICTEMMSAD’05 13/6-2005

Approach to evaluation

Identified main criteria based on Vital experiences and the quality framework approx. 70 requirements derived from the general framework,

evaluated according to relevance to Vital 32 requirements found sufficiently relevant to use in the evaluation

Identify short-list of languages to evaluate Evaluate languages based on identified criteria

Analytically Empirically (based on modeling of cases using the same

independent modeling tool (METIS)) Getting feedback from Vital on evaluations as we went along

Evaluations on a 0-3 scale on each criteria

8ICTEMMSAD’05 13/6-2005

Short-list of languages

BPMN (Business Process Modeling Notations) UML Activity diagrams EEML (Extended Enterprise Modeling Language)

9ICTEMMSAD’05 13/6-2005

BPMN - BPD

10ICTEMMSAD’05 13/6-2005

UML Activity diagrams

11ICTEMMSAD’05 13/6-2005

EEML

12ICTEMMSAD’05 13/6-2005

Overall resultsNo. Requirement description UML AD BPMN EEML

1.1 The language should support the following concepts(a) processes, that must be possible to decompose(b) activities(c) actors/roles(d) decision points

(e) flow between activities, tasks and decision points

3 3 3

1.2 The language should support (a) system resources (b) states 2 2 3

1.3 Basic control patterns 3 3 3

1.4 Advanced branching and synchronization patterns 0 0,5 3

1.5 Structural patterns 0 1,5 1,5

1.6 Patterns involving multiple instances 1,5 1,5 2

1.7 State based flow patterns 1 1 2

1.8 Cancellation patterns 3 3 3

1.9 Extension mechanisms to fit the domain 3 1 1

1.10 Elements in the process model must be possible to link to a data/information model 3 1 3

1.11 Hierarchical models 3 3 3

22,5 20,5 27,5

13ICTEMMSAD’05 13/6-2005

2.1 The language must be easy to learn, preferably being based on a language already being used in the organization

2 3 1

2.2 Appropriate level of abstraction 3 3 3

2.3 Concepts should be named similarly as it is in the domain 1 3 2

2.4 Intuitive representation to the stakeholders 2 2 2

2.5 Good guidelines for the use of the language 2 2 1

4.1 Easy diff. between different concepts 3 3 2

4.2 Number of concepts should be reasonable 3 3 1

4.3 The language should be flexible in precision 1 2 3

4.4 Easy to differentiate between the different symbols in the language 2 2 1

4.5 The language must be consistent. 3 3 3

4.6 One should strive for graphical simplicity 3 2 1

4.7 Grouping of related statements 1 1 2

5.1 The language should have a formal syntax 3 3 3

5.2 Formal semantics 1 3 2

5.3 Generate BPEL –documents from the model 2 3 0

5.4 Represent web-services in the model 1 3 1

5.5 Automatic execution and testing 1 3 2

6.1 The language must be supported by available tools. 3 3 1

6.2 Traceability between the process model and any automated process support system 2 3 1

6.3 Models that can improvement the quality of the process. 1 1 1

6.4 The language should support the development of models that help in the follow-up of separate cases

1 1 2

Sum 63,5 72,5 63,5

Sum without technical actor appropriateness 55.5 57,5 55,5

Sum without participant language knowledge appropriateness 53,5 59,5 53,5

14ICTEMMSAD’05 13/6-2005

Concluding remarks The framework found useful after specializing it to specific goals of the

organization Overly simplistic valuation ?

Weighting of importance (base more on metrics for e.g. complexity) Weighting between expressiveness, learnability, comprehension, technical

and organizational appropriateness

On later uses use several valuation schemes in parallel Also include evaluation of the meta-model and notation guides as models

Useful to first focus on the language, but language quality is only a mean to achieve model quality. Through including organizational appropriateness, tool-support and appropriate techniques to support the development of high-quality models on all levels is partly included also at this level

15ICTEMMSAD’05 13/6-2005

Assessing Business Process Modeling Languages Using a Generic Quality Framework

Anna Gunhild Nysetvold*

John Krogstie *,§

IDI, NTNU * and SINTEF§

Norway