UGent MIS research seminar June 2015

1/33

FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION

www.janclaes.info

Jan ClaesSupervisors UGent : Geert Poels & Frederik Gailly

Supervisors TU/e : Paul Grefen & Irene Vanderfeesten

Investigating the process of process modeling and its relation to modeling quality

The Role of Structure Serialization

2/33


www.janclaes.info

CHAPTER 1 – INTRODUCTIONResearch gaps

3/33www.janclaes.info

ContextIncreasing complexity of organizations

(globalization, customization, cost-effectiveness, …)Process orientation

(efficiency, responsiveness, differentiation)Process models

(representing process steps and execution constraints)Process of Process Modeling

(translate mental image of process into formal model)


Research gaps GAP 1. Need for accurate measurements GAP 2. Need for detailed, yet cognitive effective

visualizations GAP 3. Knowledge about how people construct

process models (=PPM) GAP 4. Knowledge about relation between PPM

and model quality GAP 5. Need for practical process modeling

methods GAP 6. Knowledge about process modeling

challenges

5/33


www.janclaes.info

CHAPTER 2 – VISUALIZATIONPPMChart


Study 1 – VisualizationCurrent techniques

Too high-level (Modeling Phase Diagrams) Not cognitive effective (Dotted Chart)

Design method 9 principles of cognitive effective visualization

Evaluation method Qualitative evaluation with 6 academic researchers Modeling pattern discovery


Study 1 – VisualizationCognitive effective visualization design principles

Visual expressiveness(maximal use of graphical variables)

Perceptual discriminability(visual matches conceptual distance)

Graphic economy(maximal six values per variable)

Dual coding(combine graphics with text)

Semiotic clarity(exactly one symbol per exactly one concept)

Semantic transparency(intuitiveness through natural mapping)

Complexity management(modularization and hierarchical structuring)

Cognitive integration(easy integration with other charts/models)

Cognitive fit(fit with task and user)


Study 1 – VisualizationPPMChart

CREATE_ACTIVITY CREATE_START_EVENT CREATE_END_EVENT CREATE_AND CREATE_XOR CREATE_EDGE MOVE_ACTIVITY MOVE_START_EVENT MOVE_END_EVENT MOVE_AND MOVE_XOR DELETE_ACTIVITYDELETE_START_EVENT DELETE-END_EVENT DELETE_AND DELETE_XOR DELETE_EDGE NAME_ACTIVITY RENAME_ACTIVITY NAME_EDGE RENAME_EDGE

Start event Edge Activity

Gateway

Edge

Activity

Edge

Edge

Activity

Edge Gateway

Edge

7298

9

32

14

30

31

10

3356

34

time

mod

el e

lem

ents


FastmodelingSlow

modelingInitialdelayMany

pauzesFewelementsMany

elements No(separate)lay-outing

Quicklay-outingDedicated

lay-outingphase

Continuouslay-outingUnpaired

eventcreation

Pairedevent

creationNo pauzes

SerializationPaired

gatewaycreation

Delayededge

creationChunked

modeling

Study 1 – Visualization

“Modeling patterns”

Based on dataset of 357 unique modeling executions

10/33


www.janclaes.info

CHAPTER 3 – EXPLORATIONRelation with quality


Study 2 – ExplorationRelation between modeling patterns and

process model qualityExploration method

Compare PPMCharts with process models Discover links

Evaluation method Measure definition Quantitative data collection T-tests


Study 2 – Exploration

Fastmodeling

Slowmodeling

Quicklay-outing

Dedicatedlay-outing

phase

Continuouslay-outing

Serialization

Chunkedmodeling

Structuredness Movement Speed



Study 2 – ExplorationConjecture 1: Structured modeling

results in understandable models

Conjecture 2: A high number of move operations results in less understandable models

Conjecture 3: Slow modeling results in less understandable models


Structuredness• MaxSimulBlock• PercNumBlockAsAWhole

Speed• TotTime• TotCreateTime

Movement• AvgMoveOnMovedElements• PercNumElementsWithMoves

Study 2 – ExplorationMeasurement

Model quality• Perspicuity

a model that is unambiguously interpretable and can be made sound with only small adaptations based on minimal assumptions on the modeler’s intentions with the model



T-testt=-2,231 (p=0,028)

T-testt=2,199 (p=0,030)




T-testt=-1,984 (p=0,049)

T-testt=0,457 (p=0,648)




T-testt=-2,183 (p=0,031)

T-testt=2,505 (p=0,014)


18/33


www.janclaes.info

CHAPTER 4 – THEORISATIONStructured Process Modeling Theory

(SPMT)


Study 3 – TheorizationExplanatory theoryTheory building method

6 observations, 3 impressions (induction) Explanation via existing theories (deduction)

Evaluation method Assessment of novelty, parsimony, consistency,

plausibility, credibility, and transferability Inconclusive empirical results, but open-world

assumption


Study 3 – Theorization

Combined

Flow-oriented Aspect-oriented

Undirected

“Modeling styles”



Study 3 – Theorization Observation 1. Almost all modelers paused frequently

during the modeling process Observation 2. A large group can be categorized as

“flow-oriented process modeling” Observation 3. A smaller group can be categorized as

“aspect-oriented process modeling” Observation 4. Another large group used a combination

of both former styles Observation 5. Another small group can be categorized

as “undirected process modeling” Observation 6. The “undirected” sessions lasted longer

than the other approachesBased on dataset of 118 unique modeling executions


Study 3 – Theorization Impression 1. Modelers need serialization of the modeling

process to deal with its complexity Impression 2. Structured serializing of the modeling

process helps avoiding ‘mistakes’

Impression 3. Structured serializing does not support every

modeler to avoid ‘mistakes’ to the same extent



Study 3 – TheorizationCognitive Load Theory

Working memory capacity is limited Working memory overload causes decrease in

• Effectiveness (i.e., more mistakes)• Efficiency (i.e., more time and effort)• Learning

Cognitive Fit Theory Load is lower when there is a fit

• Between representation, tool or strategy on the one hand• And task or modeler on the other hand



A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term+ +–

input material representation fit

working memory capacity

extraneous cognitive load germane cognitive load

cognitive schema construction

process model quality overall construction time

cognitive overload

intrinsic cognitive load

+++

+++

–

task complexity+

prior knowledge– –

– –



A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B

learning style

degree of serialization

adopted serialization style

field-dependency need for structure

– +

course of intrinsic cognitive load for process modeling phases

course of intrinsic cognitive load for aggregation phases

course of cognitive overload

course of intrinsic cognitive load for strategy building phases

+ + +

serialization style fitstructuredness of serialization– – – –

1 2 3


Study 3 – TheorizationNovelty

(uses existing theories in fundamental new way)Parsimony

(11 constructs, 15 associations)Consistency

(can explain additional observations)Plausibility

(accurate and profound explanation)Credibility

(building blocks are established theories)Transferability

(problem solving in general)Falsifiability

(inconclusive, but open-world assumed)Utility

(only on longer term)

Consistency based on dataset of 143 unique modeling executions

27/33


www.janclaes.info

CHAPTER 5 – CONCLUSIONSummary & Future work


Studies

Research Cycle 3

Exploration

Research Cycle 1Structured Process Modeling Theory

(SPMT)

RC6SPMT

measures RC5Cognitive measures

RC4Perspicuity

Engineering Cycle 1PPMChart RC2

Modeling styles

RC 7. Design validationRC 7. Research design

EC 2. Problem investigation

RC 3. Problem investigationRC 3. Research design

RC 3. Design validation

RC 3. Evaluation

RC 4. Evaluation

RC 4. Problem investigationRC 4. Research designRC 4. Design validationRC 4. Research

RC 3. Research

EC 1. Problem investigationEC 1. Solution design

EC 1. Design validationEC 1. Implementation

EC 1. Evaluation

RC 1. Problem investigation

RC 2. Problem investigationRC 2. Research designRC 2. Design validationRC 2. ResearchRC 2. Evaluation

RC 1. Research designRC 1. Design validation

EC 2. Evaluation

RC 8. EvaluationRC 8. Research





EC 2. Design validationEC 2. Solution design






RC 5. Design validation RC 5. Research designRC 5. Problem investigation

EC 2. Implementation

EC2Structured Process Modeling Method

(SPMM)

RC8Training

RC7Influenceability

of method


StudiesStudy 1. Visualization• EC1. How can the operations of the process of process modeling

be presented in a cognitive effective and efficient way?• RC2. How do people construct process models in terms of

modeling styles?

PPMChartResearch instrument

(visualization)

Study 3. Theorization• RC1. Why do people struggle with the complexity of process

modeling?• RC2. How do people construct process models in terms of

modeling patterns?

SPMTTheory – type II

(explanation)

Study 2. Exploration• RC3. How are process and product of modeling related? • RC4. How to measure (syntax) errors with cognitive origin?• RC2. How do people construct process models in terms of

modeling patterns?

Process vs. productConjectures(exploration)

GAP 2

GAP 3

GAP 6

GAP 3

GAP 4GAP 1

GAP 3


Future work

Study 5. Tool support• EC3. How to support measurement of cognitive profile?• EC4. How to support measurement of modeling effectiveness

and efficiency?• EC5. How to support the SPMM

SPMToolTool support

(implementation)

GAP 1GAP 1

GAP 5

Study 4. Method• EC2. How to create process models in an effective and efficient

way?• RC7. Is it possible to change a modeler’s approach towards

process modeling?• RC8. How to transform the SPMT into a prescriptive theory?

SPMMPractical method

(prescription)

GAP 5

GAP 6

GAP 5


Key publicationsPublications in international journals

Indexed by Web Of Science• J. Claes, I. Vanderfeesten, J. Pinggera, H.A. Reijers, B. Weber, G. Poels, A visual analysis

of the process of process modeling, Information Systems and e-Business Management, Vol 13(1), p. 147-190, 2015.

Under review• J. Claes, I. Vanderfeesten, F. Gailly, P. Grefen, G. Poels, The Structured Process

Modeling Theory (SPMT) A cognitive view on why and how modelers benefit from structuring the process of process modeling, resubmitted after revision to Information Systems Frontiers.


Key publicationsPublications in international conference

proceedings Indexed by Web Of Science

• J. Claes, I. Vanderfeesten, H.A. Reijers, J. Pinggera, M. Weidlich, S. Zugal, D. Fahland, B. Weber, J. Mendling, G. Poels, Tying Process Model Quality to the Modeling Process: The Impact of Structuring, Movement, and Speed, Proc. BPM '12, LNCS 7481, Springer, 2012, p. 33-48.

• J. Claes, I. Vanderfeesten, J. Pinggera, H.A. Reijers, B. Weber, G. Poels, Visualizing the Process of Process Modeling with PPMCharts, Proc. BPM '12 Workshops, LNBIP 132, Springer, 2012, p. 744-755.

• J. Claes, F. Gailly, G. Poels, Cognitive Aspects of Structured Process Modeling, Proc. CAiSE '13 Workshops, LNBIP 148, Springer, p. 168-173, 2013.

33/33


www.janclaes.info

Thanks for your attention! Do you have any questions?

Jan [email protected]

http://www.janclaes.infoTwitter: @janclaesbelgium

UGent MIS research seminar June 2015

Business

Transcript of UGent MIS research seminar June 2015