Rapid Performance Modeling by transforming Use Case Maps to Palladio Component Models

Christian Vogel (KIT), Heiko Koziolek (ABB), Thomas Goldschmidt (ABB), Erik Burger (KIT)

© ABB Group Slide 1

MotivationPerformance Modeling at ABB

Industrial Automation Domain Efficient control/data flow

crucial for real-time constraints

Design sketches on whiteboards

End-to-end latency as a requirement

Challenges How to reason on

complex control/data flows?

How to break down global response time requirements to performance budgets for software components?

© ABB Group Slide 2

Sketchof a simple webserver

Solution ApproachUCM2PCM

Palladio Component Model (PCM) allows consideration and prediction of performance from the beginning

But

Control flow and structure in different views

PCM models are difficult to create for persons from industrial automation

Thus

Use Case Maps (UCM) as new frontend for PCM

Transformation from UCM to PCM

© ABB Group Slide 3

Related WorkModel transformations in SW Performance Engineering

Performance engineering C. Smith and L.Williams, Addison-Wesley (2002)

Performance Solutions: a practical guide to creating responsive, scalable software

D. Menasce, L.W. Dowdy, V. Almeida, Prentice Hall (2004)Performance by design: computer capacity planning by example

Survey of approaches for performance predictions S. Balsamo, A. Di Marco, P. Inverardi, and M. Simeoni, IEEE TSE (2004)

Model-based performance prediction in software development: A survey

H. Koziolek, Elsevier Performance Evaluation (2010)Performance evaluation of component-based software systems: A survey

Transformation of UCM into Layered Queuing Networks (LQN) D. B. Petriu (2001)

Layered software performance models constructed from use case map specifications

Use of UML Marte Profile for creating performance models M. Woodside (2007)

From annotated software designs (UML SPT/Marte) to model formalisms© ABB Group Slide 4

FoundationsUse Case Maps

Scenario-based modeling language

Part of the User Requirements Notations (URN)

Intuitive and easy to learn

Graphical editor exists*

Allows specification of performance annotations

© ABB Group Slide 5

* http://lotos.site.uottawa.ca/ucm/bin/view/ProjetSEG/WebHome

UCM2PCMArchitecture

© ABB Group Slide 6

UCM2PCMPerformance Annotations added to UCMs

© ABB Group Slide 7

Loop IterationCount Resource Demands

Component AllocationBranch Probabilities

Scenario Workload

UCM2PCMMapping Approach

© ABB Group Slide 8

UCM Path

Component

Responsibility

UCM2PCMMapping Approach

© ABB Group Slide 9

Return Call

Component Call

All calls

UCM2PCMMapping Approach

© ABB Group Slide 10

SystemRequiredCall

SystemProvidedCall

ParentCall

UCM2PCMMapping Approach

© ABB Group Slide 11

Usage Model

UCM2PCMLimitations

Limited Input Assistance Performance Annotations need to be added as key-value pairs

Limited PCM feature support Variables & Guarded Branches are not supported

No Customized ResourceEnvironment supported

Duplicated SEFFs For every call of a function a new SEFF is created

© ABB Group Slide 12

EvaluationGoals

Evaluation of Accuracy Apply UCM2PCM to three test systems

Validate the correctness of the PCM models resulting from the UCM2PCM transformation

Measure and check the accuracy of the produced SimuCom performance results

Evaluation of Usability Tutorial & Assignment for UCM2PCM

Survey among developers for getting feedback about the UCM2PCM tool

© ABB Group Slide 13

EvaluationTested Models

Media Store Plain Java web application for storing and retrieving

media files.

SPECjAppServer Industry-standard benchmark, designed to measure the

performance of application servers conforming to the Java EE 5.0 or later specifications.

Business Reporting System Loosely modeled after a management information

system, formerly analyzed at Carlton University. Users can retrieve live business data from the system and run statistical analyses.

© ABB Group Slide 14

EvaluationBusiness Reporting System UCM

© ABB Group Slide 15

EvaluationAccuracy: Original vs. transformed UCM model

Transformation successfully bridged gap between UCM and PCM

Simulation result difference to reference model below 15% and below 5% in most cases

Accuracy is seen sufficiently for early reasoning of different design alternatives

© ABB Group Slide 16

EvaluationUsability: qualitative user survey

Main Advantages

Comprehensibility of model is high, also for non-experts

Fast(er) modeling

Modeling complex systems is seen critical with UCM2PCM

Results are only hints, without statistical relevance

© ABB Group Slide 17

UCM2PCMConclusions and Future Work

Transformation successfully bridges semantic gap between UCM & PCM

High result accuracy

Mainly positive feedback by survey

Future Work Adding Input Assistance

to UCM editor

Reverse transformation from PCM to UCM

Conducting further case studies & experiments

© ABB Group Slide 18

© ABB Group April 10, 2023 | Slide 19