Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen...
-
Upload
brittany-wood -
Category
Documents
-
view
215 -
download
0
Transcript of Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen...
![Page 1: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/1.jpg)
Alignment-based Precision CheckingA. Adriansyah1, J. Munoz Gamma2,
J. Carmona2, B.F. van Dongen1,
W.M.P. van der Aalst1
Tallinn, 3 September 2012
1) Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands
2) Software Department, Universitat Politècnica de Catalunya, Spain
![Page 2: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/2.jpg)
2
Model needs to be precise
Event Log
Process Models
a da b
bc
end
c
a b
d
end
aabdabdacd
aabdc
ad
Unfitting
How to measure precision for unfitting event logs?
d
![Page 3: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/3.jpg)
3
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabdaabdacd…
Fitting TracesAlignment
Precision Metrics
![Page 4: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/4.jpg)
4
Optimal alignment: least #deviations
a da b
bc
end
Trace adab
aAlignment
d ba a b
ad
Process Model
Fitting trace, closest to the original
Deviation
Deviation
Log Moves
Model Moves
aabd Reference: W.M.P. van der Aalst, A. Adriansyah, B.F. van Dongen (2012). Replaying history on process models for conformance checking and performance analysis. WIREs Data Mining and Knowledge Discovery, 2(2), 182-192.
![Page 5: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/5.jpg)
5
Optimal alignments can be many
a da b
bc
end
Trace ad
Alignment 1 Alignment 2
Process Model
baa d
d caa d
d
Both alignments are optimal
abd or acd
![Page 6: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/6.jpg)
6
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments Precision Metrics
aabdaabdacd…
Fitting TracesAlignment
![Page 7: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/7.jpg)
7
a
b
b
a
d
24 4
2 2 2
d 2
aabdabdabd or acdaabd
1-Alignment automaton (Prefix-based)
Fitting Traces
a da b
bc
end
Process Model
1-Align Automaton
aabdabdabdaabd
![Page 8: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/8.jpg)
8
b d
b
a
ad
4 4
2 2 2
1-Alignment automaton (Prefix-based)
0c
Fitting Traces
a da b
bc
end
Process Model
1-Align Automaton
aabdabdabdaabd
Imprecision
2 2
…
…
![Page 9: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/9.jpg)
9
a22 3
db1.51.5 11
b1a
d
3
All-Alignment Automaton (Prefix-based)
c
Fitting Traces
a da b
bc
end
Process Model
All-Alignments Automaton
aabdabdabd or acd aabd
1 1
0.5 d 0.5
4 4
2 2 2
![Page 10: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/10.jpg)
10
Comparison of log automata
aabdabdadadab
Event Log
a da b
bc
end
Process Model
Without alignment
1-Align
All-AlignPre
fix-
bas
ed L
og
Au
tom
ata
aabdabdabd/acd aabd
Fitting Traces
![Page 11: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/11.jpg)
11
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabdaabdacd…
Fitting TracesAlignment
Precision Metrics
Reference: J. Muñoz-Gama, J. Carmona (2010). A Fresh Look at Precision in Process Conformance. 8th International Conference of Business Process Management (BPM): 211-226.
Use calculation
![Page 12: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/12.jpg)
12
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabdaabdacd…
Fitting Traces
Precision Metrics
Alignment
![Page 13: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/13.jpg)
13
Implementation in ProM 6
• Package: “ETConformance”
![Page 14: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/14.jpg)
14
Experiment settings
Event Log
Flower (F)
Precise (P)
Flower’ (F’)
Precise’ (P’)
Duplicate Log Combined Log
Precise-Precise (PP’)
Precise-Flower (PF’)
Flower -Precise(FP’)
Flower-Flower(FF’)
![Page 15: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/15.jpg)
15
Experiment 1: Measuring precision stability
Perfectly fitting logs and models Non-fitting logs and models
P FP’ PF’ PP’F P FF’ FP’ PF’ PP’
Values of change significantly, but not and
![Page 16: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/16.jpg)
16
Experiment 2: Sensitivity to unfitting trace
• fluctuates when traces are unfitting
Model P
0.80
0.85
0.90
0.95
1.00
0 1 2 3 4
0.00
0.20
0.40
0.60
0.80
1.00
0 1 2 3 4
Prec
isio
n
Number of Removed Events
Noise Sensitivity - P
ETC
1-Align
All-Align𝑎𝑝❑𝑎𝑝1
𝑒𝑡𝑐𝑝❑
Pre
cisi
on
#Removed events
0.000.200.400.600.801.00
0 1 2 3 4
Model PF’
Pre
cisi
on
#Removed events
0.00
0.10
0.20
0.30
0.40
0 1 2 3 4
Pre
cisi
on
#Removed events
Model FP’
0.90
0.93
0.95
0.98
1.00
0 1 2 3 4
Pre
cisi
on
#Removed events
Model PP’
![Page 17: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/17.jpg)
17
Experiment 3: Real life cases
• Using 5 logs and models from municipalities in the Netherlands
• Computation time of is much shorter than , yet provide close estimation
![Page 18: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/18.jpg)
18
Overview
aabbdadabd
a d…
Unfitting Log
Process Model
Alignment Automaton
Experiments
aabdaabdacd…
Fitting Traces
Precision Metrics
Alignment
![Page 19: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/19.jpg)
19
Conclusion and Future Work
• Pre-alignment provides more accurate precision measurements in cases where logs are non-fitting
• Applicable to models with duplicate tasks/unlogged activities
• gives close estimation to with lower computation time• Log-Model evaluation needs to consider multiple
dimension of conformance• Potential use of the technique: auditing, conformance
diagnosis, process repair
![Page 20: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/20.jpg)
20
Questions
![Page 21: Alignment-based Precision Checking A. Adriansyah 1, J. Munoz Gamma 2, J. Carmona 2, B.F. van Dongen 1, W.M.P. van der Aalst 1 Tallinn, 3 September 2012.](https://reader035.fdocuments.in/reader035/viewer/2022062423/5697bfbe1a28abf838ca2869/html5/thumbnails/21.jpg)
21
Thanks!