Performance Analysis and Monitoring Facilities in CPN Tools Tutorial CPN’05 October 25, 2005 Lisa...

Performance Analysis and Monitoring Facilities in CPN ToolsTutorial

CPN’05October 25, 2005

Lisa Wells

Oct. 25, 2005 Lisa Wells, CPN'052

Plan

Observing and controlling simulations Introduction to monitors Example: queue system Demo 1 Performance facilities Demo 2


Simulations

Simulations of CP-nets are run for many reasons: Debugging Analysis of system behavior (performance or functional) Presentation of a model to project team Communication with external programs

The usefulness of the simulations is heavily dependent on the flexibility and functionality of the simulator. Access to simulation information Stopping simulations


Accessing simulation information It is often useful to be able to

exchange information between the CPN simulator and external processes or files.

Code segments can be added to transitions for: Reading and writing in files Calculating some

performance measures Sending and receiving

information from external processes


Controlling a simulation

Simulation stop criteria Number of steps executed Dependent on model time Dependent on markings or transitions


Common functionality

These controlling and accessing activities share a common pattern: If certain conditions are fulfilled, then do something. If transition T occurs, then save information in a file. If there are no tokens on place P, then stop the simulation. If model time is greater than C, then calculate the average

number of tokens on place P. If transition T occurs, send the values of variables x, y, and

z to the animation facilities.


Current problems in CPN Tools Problems controlling and accessing simulation

information: Cannot access marking information. Net structure may have to be changed to obtain desired

functionality. If multiple transitions need to be inspected, then code

segments must be duplicated. Code segments cannot be disabled. Only low-level support, which is difficult to use.


Monitors in CPN Tools

monitor (verb) to watch, keep track of, or check, usually for a special purpose

Merriam-Webster’s Collegiate Dictionary

A monitor is a mechanism that is used to observe, inspect, control or modify a simulation of a CP-net.

Important characteristics of monitors: They can inspect the states and events of a simulation, and

take appropriate actions based on the observations. There is an explicit separation between monitoring the

behavior of a net, and modeling the behavior of the system.


Levels of monitors

Standard monitors Very easy to define Do not require users to write any code

Parameterized monitors Similar to standard monitors, but slightly more flexible, and

require some programming User-defined monitors

Very flexible, but require more programming


Kinds of monitors

Simulation breakpoint monitors Write-in-file monitors User-defined monitors Data collector monitors


Monitoring subnets

A monitor can inspect markings and occurring transitions, with variable bindings, during a simulation Zero or more places Zero or more transitions

Monitors are activated after simulation steps After every simulation step, if no transitions are monitored After every relevant transition has occurred


Monitoring functions

A monitor typically has several functions:

Initialization function Called once before a simulation

Predicate function Called after simulation steps

Observation function Called when predicate function returns true Extracts relevant data from the model

Action function Does something appropriate with the observed value

Stop function Called once after a simulation


Example: simple queue system A single server processes two kinds of jobs. If the server is busy when a job arrives, the job is added to a queue. Job inter-arrival times and server processing times are exponentially

distributed. Queuing strategy is FIFO.

ServerQueueJob arrival


Template code: Write-in-File

Monitor one place, and two transitions

fun pred (bindelem, System'Queue_1_mark : Jobs ms) =

let

fun predBindElem (Arrivals'Arrive (1, {jobs, job})) = true

| predBindElem (Server'Start (1, {jobs, job})) = true

| predBindElem _ = false

in

predBindElem bindelem

end

fun init (System'Queue_1_mark : Jobs ms) = ""


Demo 1

Monitoring tools, monitor index entries, marking menus, syntax checking, …


Performance analysis using CPN CP-nets have the potential to be used to model and

analyze the performance, i.e. quantitative aspects, of systems.

In practice, there are very few studies using CPN for performance analysis.

This is due, in part, to lack of tool support.


Performance facilities for CPN Tools High-level support for simulation-based performance

analysis is currently being implemented for CPN Tools. Monitors

Data collection during simulations Net-specific simulation breakpoints

Simulation output Managing data files Running multiple simulations

Generating a status file for multiple simulations Collecting data from independent, terminating simulations Calculating confidence intervals for performance measures


Performance analysis of the queue system Data collection monitors for calculating performance

measures, such as: Expected average delay in queue Expected average queue length Expected utilization of the server

Breakpoint monitor: Stop a simulation when 100 jobs have passed through the

queue

The system can be modeled in CPN Tools, but until now it has been very difficult to collect and process data, and to create net-specific breakpoints.


Data collector monitors

The data that is collected is numerical data, i.e. integers, infinite integers, reals.

Statistics are calculated. Data can be saved in log

files. Data can be post-

processed, e.g. plotted in graphs, after a simulation has completed.

#data counter step time

0 1 0 0

0 2 1 0

0 3 4 548

1 4 5 579

0 5 7 595

1 6 8 697

2 7 9 710

3 8 10 848


Estimating average queue delay Wish to calculate average

queue delay for the first n jobs in the queue.

When a job arrives in the system, the time of its arrival must be noted and saved.

When a job leaves the queue, its queue delay is (current time – arrival time).


Estimating average queue length Wish to calculate the time-

average queue length until n jobs have left the queue.

When calculating statistics, the length of the queue is weighted with a time interval.

Measure the length of the queue when the length changes.

The time-average queue length is equal to the area under an appropriate curve divided by the appropriate interval of time.


Estimating server utilization

Wish to calculate the utilization of the server until n jobs have left the queue.

Use an indicator variable: server is busy => 1, server is idle => 0.

Weight the values with appropriate intervals of time.

The utilization of the server is then equal to the area under an appropriate curve divided by the appropriate interval of time.


Demo

Performance analysis of the queue system…

Performance Analysis and Monitoring Facilities in CPN Tools Tutorial CPN’05 October 25, 2005 Lisa...

Documents

Transcript of Performance Analysis and Monitoring Facilities in CPN Tools Tutorial CPN’05 October 25, 2005 Lisa...