Performance Management of IT Service Processes Using a Mashup-based Approach
-
Upload
carlos-raniery -
Category
Science
-
view
213 -
download
0
Transcript of Performance Management of IT Service Processes Using a Mashup-based Approach
PERFORMANCE MANAGEMENT OF IT SERVICE PROCESSES USING A MASHUP-BASED APPROACH
Carlos Raniery Paula dos SantosSupervisor: Lisandro Zambenedetti GranvilleThesis defense – June13 rd, 2013
2
Outline
• Introduction• Boundaries of the Investigation• Productivity• Reliability• Evaluation• Conclusions and Future Work
INTRODUCTION
4
• The increasing complexity and importance of IT environments have been leading researchers to pay more attention to the ITSM area
• Several management frameworks have then been conceived to help IT service providers towards a coherent service management experience
1INTRODUCTIONCONTEXTUALIZATION
5
• In a service management operation the presence of humans in the critical path for performing work degrades performance and quality of services
• Due to its unpredictable nature, enforcing and obtaining tight performance bounds in a human-staffed organization is burdensome task
1INTRODUCTIONPROBLEM STATEMENT
6
• Automation of IT operations has been of attention for the last two decades– Ideally, a system should be self-managed without
any human intervention
• May not be feasible in some cases because of additional effort to deploy and maintain the automation infrastructure– Specially when processes are constantly changing,
are too complex to be automated, or have a limited lifetime
1INTRODUCTIONCURRENT SOLUTIONS
7
• Mashups are Web applications composed from pre-existing Web resources (e.g., interactive maps, Web services, traditional HTML pages)
• Mashups can be composed by users with limited or no programming skills, to create their own Web applications– Basic operators are employed to hide, from mashup
end-users, technical details from the composition
1INTRODUCTIONMASHUPS
8
1INTRODUCTIONMASHUPS
9
Q I. What are the major causes for a poor performance in the execution of IT Service Management processes?
Hypothesis: The employment of mashups enhances the performance of human-centered ITSM processes
1INTRODUCTIONHYPOTHESIS AND FUNDAMENTAL QUESTIONS
Q II. What methods could be employed in the development of mashups aiming at performance enhancement?
Q III. How models available in the literature can be used to assess performance improvements obtained with mashups?
10
Q I. What are the major causes for a poor performance in the execution of IT Service Management processes?
Hypothesis: The employment of mashups enhances the performance of human-centered ITSM processes
1INTRODUCTIONHYPOTHESIS AND FUNDAMENTAL QUESTIONS
Q II. What methods could be employed in the development of mashups aiming at performance enhancement?
Q III. How models available in the literature can be used to assess performance improvements obtained with mashups?
11
Hypothesis: The employment of mashups enhances the performance of human-centered ITSM processes
1INTRODUCTIONHYPOTHESIS AND FUNDAMENTAL QUESTIONS
Q I. What are the major causes for a poor performance in the execution of IT Service Management processes?
Q II. What methods could be employed in the development of mashups aiming at performance enhancement?
Q III. How models available in the literature can be used to assess performance improvements obtained with mashups?
12
Hypothesis: The employment of mashups enhances the performance of human-centered ITSM processes
1INTRODUCTIONHYPOTHESIS AND FUNDAMENTAL QUESTIONS
Q III. How models available in the literature can be used to assess performance improvements obtained with mashups?
Q II. What methods could be employed in the development of mashups aiming at performance enhancement?
Q I. What are the major causes for a poor performance in the execution of IT Service Management processes?
13
BOUNDARIES OF THE INVESTIGATION
A service is a means of delivering value to customers by facilitating outcomes customers want to achieve without the ownership of specific costs and risks
Boundaries of the InvestigationITSM Scenario2
14
Supplier CustomerService
Definition
15
Boundaries of the InvestigationITSM Scenario2
Supplier CustomerService
Service Strategy
Service Design
Service Transition
Service Operation
Continual Service Improvement
Boundaries of the InvestigationInvestigated ITSM Process2
16
Supplier CustomerService
Service Strategy
Service Design
Service Transition
Continual Service Improvement
Service Operation
Functions
• IT Operations Management
• Application Management• Technical management• Service Desk
Processes
• Event Management• Access Management• Problem Management• Incident Management• Request Fulfillment
Boundaries of the InvestigationInvestigated ITSM Process2
17
Supplier CustomerService
Service Strategy
Service Design
Service Transition
Continual Service Improvement
Service Operation
Functions
• IT Operations Management
• Application Management• Technical management• Service Desk
Processes
• Event Management• Access Management• Problem Management• Incident Management• Request Fulfillment
Boundaries of the InvestigationInvestigated ITSM Process2
18
Supplier CustomerService
Service Strategy
Service Design
Service Transition
Continual Service Improvement
Service Operation
Request Fulfillment
Dispatcher System Administrators
Requests
Boundaries of the InvestigationInvestigated ITSM Process2
19
Supplier CustomerService
Service Strategy
Service Design
Service Transition
Continual Service Improvement
Service Operation
Request Fulfillment
Dispatcher System Administrators
Requests
• Human behavior and performance are difficult to model, and consequently, to optimize
• Even if the nature of work is exactly the same, a human may execute it in a different way each time– Interrupted by external factors– Usually employ different systems– Execute multiple and complex tasks– May be overloaded
Boundaries of the InvestigationPerformance2
20
• When the natural limits are reached or exceeded, human operators can become:– A bottleneck, slowing down the process execution– Even more error-prone
Boundaries of the InvestigationPerformance2
21
Units of work performed per unit of time
Productivity
Non-defective units at the output of the process
Reliability
• When the natural limits are reached or exceeded, human operators can become:– A bottleneck, slowing down the process execution– Even more error-prone
Boundaries of the InvestigationPerformance2
22
Units of work performed per unit of time
Productivity
Non-defective units at the output of the process
Reliability
• When the natural limits are reached or exceeded, human operators can become:– A bottleneck, slowing down the process execution– Even more error-prone
Boundaries of the InvestigationPerformance2
23
Units of work performed per unit of time
Productivity
Non-defective units at the output of the process
Reliability
• The focus is on steps that can be measured through instrumentation or observation, and can be improved through redesign and partial automation
• The performance analysis covers the steps that a human follows to execute ITSM processes
• The unpredictable nature of external events that affect human behavior is not addressed
Boundaries of the InvestigationPerformance2
24
25
PRODUCTIVITY
• Inefficiencies are portions of a service management process characterized by suboptimal execution of activities
• They can appear at different levels of analysis– Lower level: inefficiencies due to the mechanical
execution involved in performing the activity– Higher level: inefficiencies due to the complexity
of the activity itself
PRODUCTIVITYINEFFICIENCIES3
• We collected descriptions of the tasks performed by a group of operators involved in a common activity and analyzed both levels of inefficiencies– Basic: context-switching, locating data, entering data– Information Management: copy/paste, consistency
checks, information lookups– Skill-dependent: retaining information, combining
information, data transformation– Synchronization: contacting a person, becoming aware
PRODUCTIVITYINEFFICIENCIES3
• These inefficiencies can be tackled by the adoption of mashups and, more specifically, by using Mashup Patterns
• Mashup Patterns are general reusable solutions to a commonly occurring problem, which:– Provide tested and proven development
improvements– Allow mashups to be developed quickly
PRODUCTIVITYMASHUPS3
PRODUCTIVITYMASHUPS3
Housing Maps Chicago Crimes
Retrieve data (houses)
Retrieve data (crimes)
Display on Google maps
Display on Google maps
PRODUCTIVITYMASHUPS3
• Candidate mashup patterns:– Alerter: periodically monitors a system of interest
on behalf of the user and sends notifications – Importer: abstracts the different methods used to
access the external data – Transform: enables the processing of certain types
of data– Displayer: presents information from multiple
sources as independent widgets
• The Keystroke-Level Model (KLM) is used to measure lower level inefficiencies– Predicts the time an expert user takes to perform a
task on a computer system– It is based on the sequence of keystroke-level
actions the user performs
PRODUCTIVITYQUANTITATIVE MODELING3
• The Complexity Model is employed to account for the higher level inefficiencies– Evolved from a methodology for quantitative
benchmarking of configuration complexity to a model of configuration activity based on goals, procedures, and actions
– Provides a set of metrics, e.g., execution, parameter, and memory
PRODUCTIVITYQUANTITATIVE MODELING3
33
PRODUCTIVITYQUANTITATIVE MODELING3
Total time
Human-computer interactions
Cognitive actions
Analyst Domain Expert
34
RELIABILITY
35
• A planned sequence of physical or mental activities that fails to obtain a result
• May be of two types:– Involuntary actions (i.e., slips and lapses): are
those that deviate from planned intentions and, thus, do not reach their goals
– Intentional actions (i.e., mistakes and violations): are performed consciously but the desired result is not achieved
RELIABILITYHUMAN ERROR4
36
• We identified a set of common error-prone activities in ITSM:– Action: are actions that change the state of the
system– Retrieval: are failures to retrieve correct
information, to be used in a further step– Checking: occur when the operator fails to check
some information– Decision: occur when the operator has to make an
explicit choice between multiple alternatives– Communication: occur when the operator fails to
pass information to another person
RELIABILITYHUMAN ERROR4
37
• Mashup patterns, can be used to cope with human errors by providing a set of proven and reusable solutions
• We introduce a new type of mashup basic operator, (i.e., Error Prevention modules):– Buffer: allows developers to specify for how long
time a specific action should be delayed before being executed
– Forcing: allows developers to introduce confirmation points in the process workflow
RELIABILITYMASHUPS4
38
• HEART is a technique used to quantify human reliability of specific tasks– It is considered the most comprehensive method in
the field of Human Reliability Assessment– Specifies the factors that can affect human
performance, thus making it less reliable (e.g., distractions, overload)
– We employed Linguistic Variables to represent the Assessed Proportion of Affect
• Event Tree Analysis is used to determine the probability of failure in a sequence of events
RELIABILITYQUANTITATIVE MODELING4
39
RELIABILITYQUANTITATIVE MODELING4
Analyst Domain Expert
Task HEP1 6.72 2.13 4.5
HEART
40
EVALUATION
41
EVALUATIONDISPATCH PROCESS5
Service Strategy
Service Design
Service Transition
Continual Service Improvement
Service Operation
Request Fulfillment
Dispatcher System Administrators
Requests
42
ProblemTicket
IncidentTicket
ChangeTicket
Low-level group(Simple tickets)
High-level group(Higher complexity
tickets)
Mid-level group(Root
Analysis, complex
problems,etc.)
Incomingdemand
Segmentation bycomplexity
Work group structure basedOn ticket complexity
Dispatcher
OtherDispatcher
EVALUATIONDISPATCH PROCESS5
43
EVALUATIONDISPATCH PROCESS5
1) Open Ticket(ETS)
2) Analyses if theTicket was misrouted
(ETS)
4) Forwards to otherteam (ETS)
5) Analyses the skilllevel to solve the ticket
(ETS)
7) Request for more resources(e-mail)
8) Import the ticket(ETS, ITS)
9) Searches for the SAWith the right skills and
Availability (ITS)
10) Talk with the SA(in person)
11) Makes theassignment
(ETS, ITS)
3) Is the ticket
correct?
6) Have enough
resources?
• Name• Description• Severity• Workload• Skills
44
EVALUATIONMASHUP-BASED DISPATCH SYSTEM5
1) Open Ticket(ETS)
2) Analyses if theTicket was misrouted
(ETS)
4) Forwards to otherteam (ETS)
5) Analyses the skilllevel to solve the ticket
(ETS)
7) Request for more resources(e-mail)
8) Import the ticket(ETS, ITS)
9) Searches for the SAWith the right skills and
Availability (ITS)
10) Talk with the SA(in person)
11) Makes theassignment
(ETS, ITS)
3) Is the ticket
correct?
6) Have enough
resources?
• Name• Description• Severity• Workload• Skills
Alerter Pattern{New Ticket}
Displayer Pattern{name, description}
Displayer Pattern{severity}
Displayer Pattern{workload, skills}
ImporterPattern
Transformer Pattern
Input operator{System Admin}
45
EVALUATIONMASHUP-BASED DISPATCH SYSTEM5
1) Open Ticket(ETS)
2) Analyses if theTicket was misrouted
(ETS)
4) Forwards to otherteam (ETS)
5) Analyses the skilllevel to solve the ticket
(ETS)
7) Request for more resources(e-mail)
8) Import the ticket(ETS, ITS)
9) Searches for the SAWith the right skills and
Availability (ITS)
10) Talk with the SA(in person)
11) Makes theassignment
(ETS, ITS)
3) Is the ticket
correct?
6) Have enough
resources?
• Name• Description• Severity• Workload• Skills
Alerter Pattern{New Ticket}
Displayer Pattern{name, description}
Displayer Pattern{severity}
Input operator{System Admin}
Displayer Pattern{workload, skills}
ImporterPattern
Transformer PatternBuffer
OperatorForcing
Operator
46
EVALUATIONMASHUP-BASED DISPATCH SYSTEM5
47
• We performed a series of time measurements among five dispatchers in a service delivery center– Using a stopwatch, we took 10 time measurements
for each assignment process and its individual tasks
EVALUATIONPRODUCTIVITY ASSESSMENT5
2 3 5 6 8 9 110
30
60
90
120
150
180
Task #
Tim
e av
g. (s
econ
ds)
48
• The KLM model was determined through observation of user interactions
5 EVALUATIONPRODUCTIVITY ASSESSMENT
Copy and Paste Operation Time (sec)
Decide to do the task M 1.35 sec
Change Window (mouse) P + K 1.3 sec
Copy text P + B + P + B 2.4 sec
Paste text P + K + H + K + K 1.9
Ttask8 = M + Fields * CopyPaste = M + Fields * (M + Change Window + CopyText + ChangeWindow + PasteText )= 1.35 + 10 * (1.35 + 1.3 + 2.4 + 1.3 + 1.9) = 83.85 sec
49
5 EVALUATIONPRODUCTIVITY ASSESSMENT
• The complexity time can be obtained by subtracting the time predicted by the KLM model
1 2 3 5 6 8 9 110
20
40
60
80
100
120
140
160Complexity
KLM
Task #
Tim
e (S
econ
ds)
50
• By interviewing dispatchers, it was possible to determine values for all the memory and decision metrics
EVALUATIONPRODUCTIVITY ASSESSMENT5
1 2 3 5 6 8 9 110
2
4
6
8
10
12
Decision
Memory Size
Memory Latency
Memory Depth
Task #
Com
plex
uty
51
5 EVALUATIONPRODUCTIVITY ASSESSMENT
• It’s required to evaluate how the model’s quality changes as new metrics are included in the evaluation
Step Added Metric R2 RMSE
1 Decision 0.01 39.09
2 Memory size 0.94 24.78
3 Memory latency 0.96 4
4 Memory depth 0.84 5.04
52
5 EVALUATIONPRODUCTIVITY ASSESSMENT
Step Added Metric R2 RMSE
1 Decision 0.01 39.09
2 Memory size 0.94 24.78
3 Memory latency 0.96 4
4 Memory depth 0.84 5.04
• It’s required to evaluate how the model’s quality changes as new metrics are included in the evaluation
53
5 EVALUATIONPRODUCTIVITY ASSESSMENT
• It was possible to predict the time (spend at each task) associated with the complexity encountered in carrying out the tasks of the assignment process
• The model can explain 96% of the time variability
1 2 3 5 6 8 9 110
10
20
30
40
50
60
70ComplexityReal
Task #
Tim
e (s
econ
ds)
54
• Once the time measures are estimated, it’s possible to evaluate productivity enhancement by using the mashup’s technology
5 EVALUATIONPRODUCTIVITY ASSESSMENT
1 2 3 5 6 8 9 110
20
40
60
80
100
120
140
160Real measurements (without mashups)Predicted by model (without mashups)Predicted by model (with mashhups)
Task #
Tim
e (S
econ
ds)
55
• It was possible to identify the most common failures in dispatch
• The number of human errors accounted is 10 for each 150 executions (~6.6%)
5 EVALUATIONRELIABILITY ASSESSMENT
56
5 EVALUATIONRELIABILITY ASSESSMENT
EPCs HEART effect
Assessed proportion of effect
Assessed effect
Channel overload x 6 Medium 3.51
Suppressing information x 9 Medium 5.02
No veracity checks x 2 Low 1.18
No means of conveying info x 8 Low 2.28
• Once the process workflow and root causes of problems are determined, the next step is the determination of Human Error Probability (HEP) of each activity in the workflow
Task 5: Analyses the skill level to solve the ticket
5 EVALUATIONRELIABILITY ASSESSMENT
HEP[final] = HEP * (3.51 * 5.02 * 1.18 * 2.28)= 0.0004 * 47.41= 0.019 = 1.9%
Failure Task HEP
Accepts misrouted ticket 3 0.002391857
Underestimates skills 5 0.019094864
Imports wrong info 8 0.022549899
Fails to detect SA 9 0.019094864
Wrong assignment 11 0.002391857 P[failure] = 0.06401= 6.4%
57
58
• The new interaction elements and mashup patterns are feasible mechanisms to reduce the occurrence of human errors
5 EVALUATIONRELIABILITY ASSESSMENT
EPCs HEART effect
Assessed proportion of effect
Assessed effect
Suppressing information x 9 Low 2.46
No veracity checks x 2 High 1.18
No means of conveying info x 8 Low 2.28
Task 5: Analyses the skill level to solve the ticket (after mashup usage)
59
5 EVALUATIONRELIABILITY ASSESSMENT
Failure Task HEP
Accepts misrouted ticket 3 0.00091324
Underestimates skills 5 0.00412552
Imports wrong info 8 0.00000000
Fails to detect SA 9 0.00091324
Wrong assignment 11 0.00476030
HEPfinal = HEP * (2.46 * 1.83 * 2.28)= 0.0004 * 10.31= 0.0041 = 0.41%
Pfailure = 0.011004691 = 1.1 %
60
• The results show the reduction from 6.4% to 1.1% in the probability of human error
5 EVALUATIONRELIABILITY ASSESSMENT
1 2 3 4 50
0.005
0.01
0.015
0.02
0.025Without mashups With mashups
Failure #
HEP
valu
es
61
• The Pearson product-moment correlation r indicate a fair and positive relationship among the variables
5 EVALUATIONCORRELATION ANALYSIS
0 50 100 150 200 250 3000.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
7.00%MashupsWithout Mashups
Execution Time (seconds)
Cum
ulati
ve E
rror
Pro
babi
lity
(per
cent
age)
62
CONCLUSIONS AND FUTURE WORK
63
• Improvements in productivity and reliability provided by the application of mashups demonstrate its viability as a means for improving IT Service Management
• The combined model allowed us to tackle lower level inefficiencies of human-computer interactions, and higher level inefficiencies of performing IT tasks and subtasks
6 CONCLUSIONS AND FUTURE WORKFINAL CONSIDERATIONS
64
• The good fit between the real measurements with the times predicted by the combined model indicates its feasibility in predicting labor costs savings
• The presented patterns and interaction elements avoided the occurrence of action and retrieval errors
6 CONCLUSIONS AND FUTURE WORKFINAL CONSIDERATIONS
65
• Inefficiencies due to the complexity and mechanical execution involved in performing the activity.
• Four groups of inefficiencies: basic, information management, skill-dependent, and synchronization.
• Errors can occur due to: attention span, memory, situation awareness, and personal resources.
• Five common error-prone activities: action, retrieval, checking, decision, and communication errors.
6 CONCLUSIONS AND FUTURE WORKFINAL CONSIDERATIONS
Q I. What are the major causes for a poor performance in the execution of IT Service Management processes?
Hypothesis: The employment of mashups enhances the performance of human-centered ITSM processes
66
• KLM provides a wealth of detail at the lower level human-computer interactions, while the Complexity Model addresses both levels of inefficiencies.
• HEART can be used to quantify the human reliability of individual tasks. ETA was used to evaluate the overall human error probability of ITSM processes.
6 CONCLUSIONS AND FUTURE WORKFINAL CONSIDERATIONS
Q II. How models available in the literature can be used to assess performance improvements obtained with mashups?
Hypothesis: The employment of mashups enhances the performance of human-centered ITSM processes
67
• Mashup patterns to tackle inefficiencies and error-prone activities in service management processes: Alerter, Importer, Transform, and Displayer (SANTOS et al., 2011).
• A new type of interaction elements, called Error Prevention modules (SANTOS et al., 2013).
6 CONCLUSIONS AND FUTURE WORKFINAL CONSIDERATIONS
Q III. What methods could be employed in the development of mashups aiming at performance enhancement?
Hypothesis: The employment of mashups enhances the performance of human-centered ITSM processes
68
• Validation of mashup patterns through additional supporting cases
• Investigate financial models to assess loss due to broken SLAs
• Automation of mashup’s development• Inclusion of confidentiality mechanisms in the
mashup’s development• Investigation of rollback and exception handling in
the mashup composition logic
6 CONCLUSIONS AND FUTURE WORKNEXT STEPS
69
• 2009:– Rafael Santos Bezerra, Carlos Raniery Paula dos ; Leandro
Márcio Bertholdo ; Lisandro Zambenedetti Granville ; Liane Margarida Rockenbach Tarouco . Um Sistema de Gerenciamento de Redes Baseado em Mashups . SBRC 2009
– Rafael Santos Bezerra, Carlos Raniery Paula dos ; Leandro Márcio Bertholdo ; Lisandro Zambenedetti Granville ; Liane Margarida Rockenbach Tarouco . Um Sistema de Gerenciamento de Redes Baseado em Mashups . Revista Brasileira de Redes de Computadores e Sistemas Distribuídos (RESD), v. 2, 2009.
6 CONCLUSIONS AND FUTURE WORKPUBLICATIONS
70
• 2010:– Rafael Santos Bezerra, Carlos Raniery Paula dos ; Lisandro
Zambenedetti Granville ; Liane Margarida Rockenbach Tarouco . On the Feasibility of Web 2.0 Technologies for Network Management: A Mashup-Based Approach . 12 th Network Operations & Management Symposium (NOMS), 2010, Osaka, Japan
– Carlos Raniery Paula dos Santos, Rafael Santos Bezerra, João Marcelo Ceron, Lisandro Zambenedetti Granville, Liane Margarida Rockenbach Tarouco. On Using Mashups for Composing Network Management Applications . IEEE Communications Magazine, Vol. 48, Issue 12, December 2010
– Carlos Raniery Paula dos Santos, Rafael Santos Bezerra, João Marcelo Ceron, Lisandro Zambenedetti Granville, Liane Margarida Rockenbach Tarouco. Botnet Master Detection Using a Mashup-based Approach . 6th IEEE International Conference on Network and Service Management (CNSM) , 2010, Niagara Falls, Canada
6 CONCLUSIONS AND FUTURE WORKPUBLICATIONS
71
• 2011:– Carlos Raniery Paula dos Santos, Rafael Santos Bezerra, João
Marcelo Ceron, Lisandro Zambenedetti Granville, Liane Margarida Rockenbach Tarouco. Identifying Botnet Communications Using a Mashup-based Approach . LANOMS, 2011, Quito, Equador
– Carlos Raniery Paula dos Santos, Winnie Cheng, Rafael Santos Bezerra, Lisandro Zambenedetti Granville, Nikos Anerousis. A Data Confidentiality Architecture for Developing Management Mashups. 12th IFIP/IEEE International Symposium on Integrated Network Management , 2011, Dublin, Ireland
– Carlos Raniery P. dos Santos, Lisandro Zambenedetti Granville, Winnie Cheng, David Loewenstern, Larisa Shwartz, Nikos Anerousis. Performance Management and Quantitative Modeling of IT Service Processes Using Mashup Patterns . 7th International Conference on Network and Services Management (CNSM 2011), 2011, Paris, France
6 CONCLUSIONS AND FUTURE WORKPUBLICATIONS
72
• 2012:– Carlos Raniery Paula Dos Santos, Lisandro Zambenedetti
Granville, Nikolaos Anerousis, David Matthew Loewenstern, Louis Jonh Percello, Winnie Cheng, Larisa Shwartz. Performance Management and Quantitative Modeling of IT Service Processes Using Mashup Patterns . US Patent Pending
6 CONCLUSIONS AND FUTURE WORKPUBLICATIONS
73
• 2013:– Carlos Raniery Paula dos Santos, Lisandro Zambenedetti
Granville, Larisa Shwartz, Nikos Anerousis, David Loewenstern. Quality Improvement and Quantitative Modeling – Using Mashups for Human Error Prevention . 13th IFIP/IEEE Symposium on Integrated Network and Service Management (IM 2013), 2013, Ghent, Belgium
6 CONCLUSIONS AND FUTURE WORKPUBLICATIONS
74
• Virtual Nodes Monitoring based on Mashup – This paper provides a mashup-based
mechanism to monitor virtualized networks.
6 CONCLUSIONS AND FUTURE WORKNEXT PUBLICATIONS
75
• A Mashup-based Solution for Botnet Mitigation – This paper proposes a modular architecture
based on the dynamic integration of pre-existing tools to achieve a more efficiently detection solution.
6 CONCLUSIONS AND FUTURE WORKNEXT PUBLICATIONS
76
• Survey– This work aims at gathering and organizing
the knowledge about mashups properties applied to multiple management scenarios
6 CONCLUSIONS AND FUTURE WORKNEXT PUBLICATIONS
77
• Automation of mashups’ development– Considering that the ITSM operators have a
strong knowledge of the process they use to perform, but may not have expertise in mashups development, the use of an automated development system may ultimately provide a significantly improved orchestration of the process
6 CONCLUSIONS AND FUTURE WORKNEXT PUBLICATIONS
78
• Overview of the Thesis– This paper will present all the performance
metrics analyzed and show the key concepts of this thesis
6 CONCLUSIONS AND FUTURE WORKNEXT PUBLICATIONS
79
Carlos Raniery Paula dos SantosOrientador: Lisandro Zambenedetti Granville
Instituto de Informática – UFRGSInf.ufrgs.br/~crpsantos
Some questions….
ComputerNetworksGroup
Thank you for your attention!
80
• [01] ISACA. Control Objectives for Information and related Technologies (COBIT). 2008
• [02] OGC. Information Technology Infrastructure Library v3 (ITIL v3). 2008
• [03] PULTORAK, D.; HENRY, C.; LEENARDS, P. MOF 4. 0: a pocket guide: (microsoft operations framework). [S.l.]: Van Haren Publishing, 2008. (Van Haren Series)
• [04] MILLIKEN, K. R.; CRUISE, A. V.; ENNIS, R. L.; FINKEL, A. J.; HELLERSTEIN, J. L.; LOEB, D. J.; KLEIN, D. A.; MASULLO, M. J.; VANWOERKOM, H. M.;WAITE, N. B. YES/MVS and the automation of operations for large computer complexes. IBM System Journal, [S.l.], v.25, p.159–180, June 1986.
• [05] CANDEA, G.; KICIMAN, E.; KAWAMOTO, S.; FOX, A. Autonomous recovery in componentized Internet applications. Cluster Computing, Hingham, MA, USA, v.9, p.175–190, April 2006.
References
81
• [06] KELLER, A.; HELLERSTEIN, J. L.; WOLF, J. L.; WU, K. L.; KRISHNAN, V. The CHAMPS system: change management with planning and scheduling. In: NETWORK OPERATIONS AND MANAGEMENT SYMPOSIUM, 2004. NOMS 2004. IEEE/IFIP, 2004. Anais. . . [S.l.: s.n.], 2004. v.1, p.395–408 Vol.1.
• [07] LIKER, J. The Toyota way: 14 management principles from the world’s greatest manufacturer. [S.l.]: McGraw-Hill, 2004.
• [08] OGC. Information Technology Infrastructure Library v3 (ITIL v3). 2008.
• [09] Michael Ogrinz. Mashup Patterns: Designs and Examples for the Modern Enterprise. 2009. ISBN 978-0321579478
References
82
• [10] CARD, S. K.; NEWELL, A.; MORAN, T. P. The Psychology of Human-Computer Interaction. Hillsdale, NJ, USA: L. Erlbaum Associates Inc.,2000
• [12] BROWN, A. B.; HELLERSTEIN, J. L. An approach to benchmarking configuration complexity. In: IN PROCEEDINGS OF THE 11TH ACM SIGOPS EUROPEAN WORKSHOP, 2004. Anais. . . [S.l.: s.n.], 2004.
• [13] DIAO, Y.; KELLER, A. Quantifying the Complexity of IT Service Management Processes. In: DSOM, 2006. Anais. . . [S.l.: s.n.], 2006. p.61–73
• [14] DIAO, Y.; KELLER, A.; PAREKH, S. S.; MARINOV, V. V. Predicting Labor Cost through IT Management Complexity Metrics. In: INTEGRATED NETWORK MANAGEMENT, 2007. Anais. . . [S.l.: s.n.], 2007. p.274–283
References
83
• [15] REASON, J. Human Error. [S.l.]: Cambridge [England] ; New York : Cambridge University Press, 1990. xv, 302 p., 1990.
• [16] KIRWAN, B. The validation of three human reliability quantification techniques THERP, HEART and JHEDI: part 1 -ttechnique descriptions and validation issues. Applied Ergonomics, [S.l.], v.27, n.6, p.359 – 373, 1996
References
84
Browser
Browser
asdasdasdasdasdasdasdasdasdasdasdasdasdasdasdasdasdasdadsasdasdadsasda
asdasdasdasdasdasdasdasdasdasdasdasdasdasdasdasdasdasdadsasdasdadsasda
Mashup System
Developer
End User Content Providers
1INTRODUCTIONMASHUPS
85
RELIABILITYMASHUPS4
Adaptation Adaptation
Operation(merge)
Forcing
Buffer
Adaptation
External System
Mashup System
86
PRELIMINARY EVALUATIONDISPATCH PROCESS5
87
• The focus is on the Request Fulfillment process– Responsible for carrying out service requests, and
that interfaces with Service Desk, Incident Management, and Change Management
• This process includes two types of human operators:– System administrators: technical personnel with
knowledge to resolve specific requests– Dispatchers: responsible for monitoring new
requests, dispatching the requests to the appropriate SA, and monitoring compliance with SLAs
PRELIMINARY EVALUATIONDISPATCH PROCESS5
88
• Interaction Components:– Visual: represent way data can be displayed, such
as map, tables, and trees– Control: represent basic programming logic, such
as loops, and conditions– Operation: perform operations over retrieved
information, such as filtering, merging, and arithmetic operations
– Adaptation: represent external resources, these are created based on wrapper meta-information stored
– Reuse: this type of component represent existing mashups, enabling their reuse in other masups
Mashup System
89
• Effectiveness: how to match the customers requirements and what the service provides in fact
• Cost: may be defined in SLAs• Security: probability of information leakage
Other metrics
90
• Checklists are increasingly being used in Scottish hospitals, for example in pre-operative settings
• Minimize staff interruptions and distractions• Prevent errors: procedures, training, UI design
(allow only valid choices)• Recover errors: undo capability, confirmation
Other system improvements
91
Mashup Patterns
92
Gesture Time
K Keying 0.2 sec
B Holding/Releasing key 0.1 sec
P Pointing 1.1 sec
H Homing 0.4 sec
M Mentally Preparing 1.35 sec
KLM
93
• KLM model: provides a wealth of detail at the lower level of human-computer interactions
• Complexity model: besides it addresses both levels of inefficiencies, we discard all the complexity metrics except the memory and decision metrics, which capture higher level potential inefficiencies not addressed by KLM
Total time spend by a human
Human-computer interactions
Cognitive actions
Quantitative Modeling
94
• The approach can be summarized in three steps:– Assessing the complexity and timing a baseline
scenario– Construction of the regression model and its quality
evaluation– Employing the model to predict labor costs
Complexity Model