Analyzing performance of concurrent usage scenarios using software architecture analysis

Analyzing performance of concurrent usage scenarios using software architecture analysis

Tarja Kauppi & Anu Purhonen, VTT Technical Research Centre of Finland

16th International Conference on Software & Systems engineering and their Applications

2nd - 4th December 2003, Paris

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

19.04.23 Copyright Software Architectures Group, Tarja Kauppi 2

Outline

• Motivation

• Available software architecture -based performance analysis methods

– Typical process

– Purpose

– Requirements

– Methods

• Method overview

• Example analysis

• Conclusions



Motivation

• Hardware capacities have grown, however software has become more complex.

• Users have greater freedom to start many applications.

• Traditional approach: – first implement and then test– if performance problems, then modify

• Software architecture -based approach:

– design the software system so that when it is ready it meets its performance objectives -> no later modifications

• Software architecture -based approach is not systematically used in the industry

– ad-hoc methods ->results are not comparable and they depend on the experience of the software architect

RequirementsAnalysis

RequirementsAnalysis

FunctionalArchitectureFunctional

Architecture

PreliminaryDesign

PreliminaryDesign

Detail DesignDetail Design

CodingCoding

Unit TestingUnit Testing

Integration Testing

Integration Testing

Maintenance and OperationMaintenance and Operation

Software architecture -based performance engineering

Traditional performance engineering



Available software architecture -based performance analysis methods: typical process

SoftwareArchitecture

SoftwareArchitecture

TransformationTransformation Performance Model

Performance Model

AnalysisAnalysis

Timing Information

Timing Information

Feedback



Available software architecture -based performance analysis methods: purpose

• predicting the performance of the system,

• guaranteeing that performance goals are met,

• comparing different architectural choices from performance point of view,

• finding bottleneck resources,

• identifying potential timing problems,

• aid when deciding if the architecture is worth implementation



Requirements

1. easy to use and integrate in the software development process,

2. simple modelling approach,3. tool support available,4. good instructions and case studies available of applying the

method in practice,5. should help in outlining the problem area



Available software architecture -based performance analysis methods

Method Modelling approach

Properties

PASA Software execution graph and QNM

- provides framework to whole assessment process - two books supporting the usage of the method - SPEED tool

LQN LQN - developed for modelling distributed and/or concurrent systems - good instructions available - LQN simulator tool

RMA - - does not provide support for the whole assessment process - provides only the principles and rules for analysing the schedulability of a system - RapidRMA, TimeWiz

QNM&LTS QNM - has not gained much popularity - not many case studies available - no tool support

CPN CPN - knowingly applied only once at the software architectural level - solving the model requires a Design/CPN tool - modelling approach quite complicated

SPA SPA, LOTOS - formal approach - has not gained much propularity - several tools available

Method Modelling approach

Properties

PASA Software execution graph and QNM

- provides framework to whole assessment process - two books supporting the usage of the method - SPEED tool

LQN LQN - developed for modelling distributed and/or concurrent systems - good instructions available - LQN simulator tool

RMA - - does not provide support for the whole assessment process - provides only the principles and rules for analysing the schedulability of a system - RapidRMA, TimeWiz

QNM&LTS QNM - has not gained much popularity - not many case studies available - no tool support

CPN CPN - knowingly applied only once at the software architectural level - solving the model requires a Design/CPN tool - modelling approach quite complicated

SPA SPA, LOTOS - formal approach - has not gained much propularity - several tools available

- PASA methodas a overal framework- LQN for modelling- RMA principles for performance improvements

- PASA methodas a overal framework- LQN for modelling- RMA principles for performance improvements



Method Overview

Software ArchitectureOverview

Software ArchitectureOverview

Critical Use CasesCritical Use Cases

Key Performance Scenarios

Key Performance Scenarios

Performance Objectives

Performance Objectives

Architecture DetailsArchitecture Details

Performance Modellingand Analysis

Performance Modellingand Analysis

Performance ImprovementsPerformance Improvements

-> high level understanding about the architecture

-> identification of use cases that are important from performance point of view

-> identification of the scenarios that are executed frequently or the ones that are critical from user’s perception of performance

-> at least one quantitative objective is identified to each key performance scenario and condition under which the objectives should be met is defined

-> related architectural components

-> drawing a LQN model-> calculating utilisation and residence time-> comparing calculated values with performance objectives

-> applying performance anti-patterns as an aid to remove bottlenecks from the architecture-> applying RMA principles to guarantee that the most critical tasks will meet their deadlines



Software Architecture Overview

BearerProtocolsBearer

Protocols

DSP Software

DSP Software

Symbian SoftwareSymbian Software



Critical Use Cases, Key Performance Scenariosand Performance Objectives

Mobile Phone

Streaming Content Server

Video stream40 packets/s

Audio stream 10 packets/s

Multimedia content 1 packet/s

Multimedia Service Centre

(MMSC)

Critical Use Cases:• Multimedia streaming

• Multimedia message (MMS)

Key Performance Scenarios:• receiving video stream packets• receiving audio stream packets• receiving MMS content packets

Performance Objectives:-> max response time 25 ms/packet -> max response time 100 ms/packet-> max response time 1000 ms/packet



Architecture Details


Protocols

DSP Software

DSP Software

IP StackIP Stack

UDP StackUDP Stack

RTP/RTCP StackRTP/RTCP Stack

Multimedia Streaming Player



Protocols

IP StackIP Stack

TCP StackTCP Stack

HTTP StackHTTP Stack

MMS ServerMMS Server

Messaging ServerMessaging Server

Video and audio stream packets MMS content packets

0.3 ms

0.35 ms

0.2 ms

1.0 ms

0.3 ms

= 1.85 ms

0.35 ms

2.0 ms

19.6 ms

27.0 ms

= 49.25 ms



Performance modelling

ProcessorProcessor

StreamingStreamingAudio10 packets/s

Audio10 packets/s

Video40 packets/s

Video40 packets/s MMSMMSContent

1 packet/sContent

1 packet/s

RTP/RTCP Stack0.2 ms


UDP Stack

0.35 ms

UDP Stack

0.35 ms


1.0 ms


1.0 ms

IP Stack0.3 ms

IP Stack0.3 ms

HTTP Stack2.0 ms

HTTP Stack2.0 ms

MMS Server19.6 ms

MMS Server19.6 ms

TCP Stack0.35 ms

TCP Stack0.35 ms





Analysis

• Performance objectives are met on average, however performance problems may occur in worst-case situation.

– > Changes are not necessarily needed to the system.

• Note! The formulas are applicable if

– tasks are scheduled according to first-come-first-served or priority scheduling policy

– if the system can be considered to be fast enough to handle arrivals

Utilisation:

Utilisation upper limit for n tasks:

Residence time:

Utilisation:

Utilisation upper limit for n tasks:

Residence time:

Performance Metric Value Maximum

Utilization (U) 14.2 % 77.9 %

Residence time (R) for video stream 2.16 ms 25 ms

Residence time (R) for audio stream 2.16 ms 100 ms

Residence time (R) for MMS content 57.38 ms 1000 ms

)12()( /1 nnnU

XSU

U

SR

1



Performance improvements

• The conditions could be assumed to be met in this case, so by assigning priorities as follows performance objectives should be met in the worst-case situation.

– Video stream (40 packets/s)

-> highest priority

– Audio stream (10 packets/s)

-> next highest priority

– MMS content (1 packet/s)

-> lowest priority

RMA scheduling principle:RMA scheduling principle:

• if utilisation is less than utilisation upper limit,• if tasks are periodic,• they are not synchronised,• they do not suspend themselves during execution,• and they are capable of being preempted by higher priority tasks,• then priorities can be assigned in a way that deadlines are met even in the worst-case situation.

Tasks with highest arrival rate-> highest priorityTasks with next highest arrival rate-> next highest priority...

RMA scheduling principle:RMA scheduling principle:

• if utilisation is less than utilisation upper limit,• if tasks are periodic,• they are not synchronised,• they do not suspend themselves during execution,• and they are capable of being preempted by higher priority tasks,• then priorities can be assigned in a way that deadlines are met even in the worst-case situation.

Tasks with highest arrival rate-> highest priorityTasks with next highest arrival rate-> next highest priority...



Conclusions

• The approach provides added value to the current software system development.

• Quantitative values produced are approximations, however the approach helps to manage the performance of a complex system, because it guides to concentration on the parts that are important from the performance point of view.

• Problems:– does the performance model illustrate the system in enough detail?

– where to obtain the timing values if nothing can be measured?

– how do made assumptions effect on performance?

• The approach probably produces best results if the same type of system has earlier been implemented -> timing estimations based on earlier measurements.

• If the type of system is developed for the first time, then this kind of quantitative analysis is not meaningful -> using performance anti-patterns as an aid is more rational, because they do not require timing information.



Questions

• More about the issue:• Kauppi, T. & Purhonen, A.

Performance analysis of concurrent usage scenarios using software architecture analysis. ICSSEA 2003, 2-4 December, Paris.

• Kauppi, Tarja. Performance analysis at the software architectural level, VTT Publications 512, Espoo 2003. http//www.vtt.fi/inf/pdf/publications/2003/P512.pdf

• email:

[email protected],

[email protected]

Analyzing performance of concurrent usage scenarios using software architecture analysis

Documents

Transcript of Analyzing performance of concurrent usage scenarios using software architecture analysis