Output Analysis for Simulation

Written by:Marvin K. Nakayama

Presented by:Jennifer BurkeMSIM 752

Outline

Performance measures Output of a transient simulation Techniques for steady-state

simulations Estimation of multiple performance

measures Other methods for analyzing

simulation output

Example

Automatic Teller Machine (ATM)

Performance Measures

Measure how well the simulation runs

Different types of simulations require different statistical techniques to analyze the results Terminating (or transient) Steady-state (or long run)

Terminating Performance Measures

Terminating simulation Simulation will finish at a given event Initial conditions have a large impact

Ex: Queue starts with no customers present

ATM example (Terminating)

Open 9:00am – 5:00pm X = # of customers using ATM in a

day E(X) P(X 500)

C = queue is empty

Output of a Terminating Simulation

Goal: calculate E(X)

Approach: n 2 i.i.d duplications

X1,X2,…,Xn

find the average of those duplications

Output of a Terminating Simulation

calculate the sample variance of X1,X2,…,Xn

and the sample standard deviation

Output of a Terminating Simulation

Central Limit Theorem

confidence interval for E(X)

Output of a Terminating Simulation

the confidence interval provides a form of error bound

Hn is the half-width of the confidence interval

ATM example (Terminating)

Expected daily withdraw within $500

ε = 500 S(n) = sample standard

deviation

Steady-state Performance Measures

Steady-state simulation Simulation that stabilizes over time

Initial condition C Fi(y|C)

Fi(y|C) → F(y) as i →

ATM example (Steady-state)

Open 24 hours a day Yi = number of customers served on

the ith day of operation E(Y) P(Y 400)

C = queue is empty

Output of a Steady-state Simulation

Case 1: discrete-time process Y1,Y2,…,Yn

estimate v, as m →

Case 2: continuous-valued time index Y(s)

estimate v, as m →

ATM (Continuous)

Y(s) = number of customers waiting in line at time s

Assume Y(s) has a steady-state Calculate v

Difficulties of Steady-state analysis

Discrete-time process

if m is large, then is a good approximation of v

Confidence interval

Simplifications to Steady-state Analysis

Multiple replications Initial-data deletion Single-replicate algorithm

Method of Multiple Replications

Estimate

r i.i.d replications, length k = m/r 10 r 30

Method of Multiple Replications

Average of jth row

Using find the sample mean

Method of Multiple Replications

Sample variance

Confidence interval

Problem with Multiple Replication Method

Simple estimation of variance

can be contaminated by initialization bias

Initial-Data Deletion

Partial solution Delete first c observations Replication mean

sample mean sample variance

confidence interval

Single-Replicate Algorithm

Single simulation of length m + c Divide the m observations into n

batches

10 n 30 Batch mean

Single-Replicate Algorithm

Sample mean

Sample variance

Confidence interval

Estimating Multiple Performance Measures

Terminating simulations

Confidence interval for each performance measure

Joint confidence interval

ATM example (Terminating)

Open 9:00am – 5:00pm μ1 = expected # of customers

served in a day μ2 = probability # served in a day is

at least 1000 μ3 = expected amount of $

withdrawn in a day

Conclusions

Basics of analyzing simulation output

Application potential is high Not state of the art Benefit Lacked comparison