Technische universiteit eindhoven 20 October 2001btheelen1 Performance Modeling in the Large: A Case...

20 October 2001 www.ics.ele.tue.nl/~btheelen 1

technische universiteit eindhoven

Performance Modeling in the Large:A Case Study

B.D. Theelen



Contents• Introduction

• Parallel Object-Oriented Specification Language (POOSL)

• Case Study: Internet Router

• Modeling the Internet Router

• Exhaustive Performance Analysis

• Performance Evaluation by Simulation

• Accuracy Analysis

• Conclusions



System-Level Design Methods• Focus on developing executable models, which

– Describe a system in the earliest phases of the design process

– Abstract from many implementation details

– Are relatively cheap to create, validate, test, debug and simulate

• Allow the analysis of system properties before the system is actually being realized in terms of hardware and software– Correctness propertiesCorrectness properties: absence of deadlock, …

– Performance propertiesPerformance properties: throughput, packet-loss, jitter, …

• Enable to take well-founded design decisions based on the obtained system property analysis results

• Require the use of a well-defined modeling languageParallel Object-Oriented Specification Language (POOSL)Parallel Object-Oriented Specification Language (POOSL)



POOSL• System-level modeling language for complex real-time

distributed hardware/software systems• Small set of very expressivevery expressive primitives• Formal (mathematically definedmathematically defined) semantics

– Probabilistic features for performance analysis

• Process part based on real-time extension of CCS– Process objects model the elementary time-related behavior– Channels enable synchronous message passing between process objects– Clusters allow for describing hierarchy

• Data part based on imperative object-oriented languages– Data objects model data structures and operations on them

www.ics.ele.tue.nl/~lvbokhov/pooslwww.ics.ele.tue.nl/~lvbokhov/poosl



Alcatel 7770 RCP (Routing Core Platform)

Internet Router640 Gb/s – 10 Tb/s

OC-3 – OC-192

8.5 kW

$ 290.000



Overview of the Internet Router

Is the specified flow-control mechanismsuitable for future product variants?

Inputs Outputs

SourcesInput Buffers Output Buffers

SinksSwitch Core

1 1

N N

1

N

N

1

N

1

1

N

M

1

M

1



System Parameters N and M• Sources NxM Source Activities

• Input Buffers N Input Dispatchers

NxN Queue Output Handlers

• Switch Core NxN Switch Activities

• Output Buffers N Queue Input Handlers

N Queue Output Handlers

• Sinks N Sink Activities

2N2 + (M+ 4) N Activities

Huge number of concurrent activitiesHuge number of concurrent activities



Scalability of the Model• Instantiating a process object is performed by drawingdrawing the object

• Modeling each component with a separate process object is not favorable for large N and M

• Initialize parameterizedparameterized number of similarsimilar concurrent activities

Initialize()()

InitializeActivity(1)().

InitializeActivity(Identity: Integer)()

par

Activity(Identity)() /* Invokes the actual Activity */

and

if (Identity < N) then InitializeActivity(Identity + 1)() fi

rap.

(Optimization: dynamic creation and termination of concurrent activities)



Executable Model• Create executable model with SHESimSHESim tool

– Freely available on: www.ics.ele.tue.nl/~mgeilen/shesim



Validation• Is the model adequateadequate for analyzing the performance metrics?

• Validate, test and debug model using SHESimSHESim tool



Formalization of Performance Properties• The flow-control mechanism should contribute to

– Minimizing packet-loss

– Minimizing buffer capacity

– Minimizing latency

– Maximizing throughput

• Performance metrics to analyze– Probability of losing packets

– Average and maximum buffer occupancy

– Average and maximum latency

– Throughput

Use POOSL formalism to extend model for performance analysisUse POOSL formalism to extend model for performance analysis



Performance Evaluation with POOSL

Formal SemanticsFormal Semantics

Markov Decision Process&

Reward Structure

UnderstandablePOOSL Model

&Extensions for Performance

Analysis

1 0.9

0.1

3

Out!Packet(p)1

S1 S2 S3 S4

External SchedulerExternal Scheduler

Discrete-Time Markov Chain & Reward Structure

Markov Chain is implicitly defined by expressive modeling language and not by handMarkov Chain is implicitly defined by expressive modeling language and not by hand



Performance Evaluation with POOSL• Analytically: Ergodic Theorem for Markov Chains

– Example: time-average buffer occupation

• Simulation: Central Limit Theorem for Markov Chains– Dependable random variables– Example: packet-loss probability

– Enables accuracy analysis with confidence intervals

Occupation Average )(

)()(

)(

)()(a.s.

1

1

SS

SS

n

ii

n

iii

Stπ

StSOccupationπ

Xt

XtXOccupation

N(0,1) σ

μ)(1

μ then )(1

if d.1a.s.

1

n

iin

ii

XPacketLostn

nXPacketLostn



Performance Evaluation by Simulation• Fast simulation with RotalumisRotalumis tool

– Freely available on: www.ics.ele.tue.nl/~lvbokhov/poosl/rotalumis

• Simulation = generation of one trace through Markov chain• How long should a simulation take to obtain accurate results?• Analyze accuracy of simulation results with confidence intervalsconfidence intervals• Automatic termination of simulation



Accuracy Analysis• For N=512 and M=128, a 99% confidence interval

[0.001046, 0.001052] is obtained for the average latency– Estimation of average latency is 0.001049 seconds

– We are 99% sure that the real average latency lies in [0.001046, 0.001052]

– An upper bound for the relative error is 1%

– Takes 0.0072 seconds of simulated real-time

– Takes 11.9 hours of simulation time

• POOSL classes for accuracy analysis of long-run averageslong-run averages– Long-run sample average: packet-loss probability, latency

– Long-run sample variance: jitter

– Long-run time average: buffer occupancy, processor utilization

– Long-run time variance: buffer occupancy

Rare events (losing a packet) may change the status of a metricRare events (losing a packet) may change the status of a metric



Conclusions• POOSL allows for a compact abstractcompact abstract description of real-life real-life

systemssystems to adequatelyadequately investigate their performanceperformance properties• Performance analysis requires extending the original model• POOSL model implicitly defines a Markov chain• Formal semantics enables computationcomputation of performance• POOSL tools offer good validationvalidation and simulationsimulation possibilities• Confidence and accuracy of simulation results should be known• POOSL library classes for accuracy analysisaccuracy analysis• Future research

– Long-run rate average (throughput) and rate variance (burstiness)– Rare events– Support for performance metric monitorsmonitors

Technische universiteit eindhoven 20 October 2001btheelen1 Performance Modeling in the Large: A Case...

Documents

Transcript of Technische universiteit eindhoven 20 October 2001btheelen1 Performance Modeling in the Large: A Case...