Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.
-
Upload
collin-roberts -
Category
Documents
-
view
224 -
download
6
Transcript of Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.
![Page 1: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/1.jpg)
Simulation &Confidence IntervalsCOMP5416Advanced Network Technologies
![Page 2: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/2.jpg)
COMP5416 Confidence - 2
School of Information Technologies
Simulation Implementation Details
• Need to store– Core variables such as event list, sim clock– State variables such as packet queue– Statistics such as packet arrived, departed
• Initialise for bootstrap– E.g. Packet arrival
• Provide terminate condition– End time or number of events to run
• Execute events from list
![Page 3: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/3.jpg)
COMP5416 Confidence - 3
School of Information Technologies
E.G.: SSQ Simulation
Start
Init
MoreEvents?
Arrvl? Dept?
Update varsSchedule next arrival
Update varsSchedule next departure
Yes
Yes Yes
No
End
No
Compute performanceDisplay stats
![Page 4: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/4.jpg)
COMP5416 Confidence - 4
School of Information Technologies
Transients in the Simulation
• Everything above assumes that the simulation data is collected once the simulation has achieved steady state, otherwise the tests are meaningless.
• There are many ways to determine this– a pragmatic approach is to ensure that the simulation runs
for sufficiently long so that the transients have negligible effect on the estimates of confidence interval.
• If data is available from only short runs (i.e. few data points), then you will need to ensure that transient data is discarded.
![Page 5: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/5.jpg)
COMP5416 Confidence - 5
School of Information Technologies
Independent Samples
• Assume we have a sequence of n data points {z1,z2,...,zn} which are known to be independent.
• Estimate mean:
• Estimate Sample Variance:
n
z
z
n
ii
1ˆ
)1(
ˆ*ˆ
2
1
2
n
znzV
n
ii
![Page 6: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/6.jpg)
COMP5416 Confidence - 6
School of Information Technologies
Confidence Interval for Independent Samples• Assume that there are enough samples so that we can use the
Central Limit Theorem (i.e. approx Gaussian distributed)• Make a hypothesis test – with (say) 95% probability, the true
mean lies within some interval• From a Gaussian distribution, the 95% confidence interval is
the estimated mean, ±1.96 estimated standard deviations• The standard deviation of the sample mean is
• Then we say that, with probability 95%, the true mean lies within the range
nV̂
n
Vz
n
Vz
ˆ96.1ˆ,
ˆ96.1ˆ
![Page 7: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/7.jpg)
COMP5416 Confidence - 7
School of Information Technologies
Estimated Mean
Confidence Interval,with probability x%,true mean lies withinthis range
Confidence Interval
![Page 8: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/8.jpg)
COMP5416 Confidence - 8
School of Information Technologies
Effect of Correlation• Simulation data is never independent – there is correlation
between successive samples. – It is always a good idea to estimate the correlation – the
autocovariance of lag k is defined as:
where is the true mean
))(()( zzzzEkR kii
z
• If the autocovariances are low, then an approximation of independence is reasonable, otherwise we need to do better.
![Page 9: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/9.jpg)
COMP5416 Confidence - 9
School of Information Technologies
Method of Batch Means
• The idea here is to group the data into batches, so that successive batches are approximately independent.
• For example, if we have n data points , group them into m batches of size p=n/m
• Calculate the m batch means• Now treat the batches as if they were independent
![Page 10: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/10.jpg)
COMP5416 Confidence - 10
School of Information Technologies
Batch Means
• Gives less bias in the estimator, at the expense of more variability in the estimator of precision. The difficult part is choosing the batch size – too large a batch produces confidence intervals which are
much larger than is justified by the actual data – too small a batch produces excessive correlation, which
produces misleadingly small confidence intervals• Method is easy to implement, and is probably the
most widely-used method of estimating confidence intervals.
![Page 11: Simulation & Confidence Intervals COMP5416 Advanced Network Technologies.](https://reader036.fdocuments.in/reader036/viewer/2022082505/56649f425503460f94c61959/html5/thumbnails/11.jpg)
COMP5416 Confidence - 11
School of Information Technologies
References
• B.D. Ripley, “Uses and abuses of statistical simulation”, Mathematical Programming, Vol 42, pp 53-68, 1988
• K. Pawlikowski, “Steady-state simulation of queueing processes: A survey of problems and solutions”, ACM Computing Surveys, Vol 22, No 2, pp123-170, June 1990
• A.M. Law and W.D. Kelton, Simulation Modelling and Analysis, McGraw-Hill, 1991