KPC-Toolbox Demonstration
description
Transcript of KPC-Toolbox Demonstration
![Page 1: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/1.jpg)
1
KPC-ToolboxKPC-Toolbox DemonstrationDemonstrationEddy Zheng Zhang, Giuliano Casale, Evgenia SmirniEddy Zheng Zhang, Giuliano Casale, Evgenia Smirni
Computer Science DepartmentComputer Science DepartmentCollege of William & College of William & MaryMary
![Page 2: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/2.jpg)
2
What is KPC-Toolbox for?
KPC-Toolbox: MATLAB toolbox Workload Traces Markovian Arrival Process (MAP)
Why MAP? Very versatile High variabilityHigh variability & temporal dependence temporal dependence in Time SeriesTime Series Easily incorporated into queuing models
Friendly Interface Departure from previous Markovian fitting tools Fit the automatically (no manual tuning)
![Page 3: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/3.jpg)
3
User Interface Requirement: Matlab installed
Input A trace of inter-event times Or a file that already stores the statistics of the trace
E.g., a file stores the moments, autocorrelations and etc
Help Information Type “help FunctionName”,
E.g., “help map_kpcfit” Website Keeps Up-To-Date Tool version
http://www.cs.wm.edu/MAPQN/kpctoolbox.html
![Page 4: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/4.jpg)
4
A Simple Example of MAP Two state jumps
1 2 00 bbaa
ccdd
D1 =
D0 =-b-d
-a-c
Time:
a bc
d
I1 I2
I3
Background Jumps Jumps With Arrivals
Arrivals:
![Page 5: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/5.jpg)
5
Challenges
How large is the MAP? MAP(n): determine n?
Which trace descriptors are important? Literature: Moments of interval times, lag-1 autocorrelation But, for long range dependentlong range dependent traces?
Need temporal dependencetemporal dependence descriptors
MAP Parameterization Construct MAP(n) with matrices D0 and D1 (2n2 – n entries)
![Page 6: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/6.jpg)
6
Example: Important Trace Statistics
1
2
First, second, third moment and lag-1 autocorrelation accurately fit
The queuing prediction ability is not satisfactory!The queuing prediction ability is not satisfactory!
Seagate Web Server Trace Queue Prediction, 80% Utilization
Fit With MAP(2)
100 101 102 103 104 105 106 10710-4
10-3
10-2
10-1
100
Pr(Q
ueue
Len
gth
> X)
X [log]
Trace /M/1 MAP(2)/M/1
![Page 7: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/7.jpg)
7
Example: Higher Order Statistics Matter
Much Better Results!Much Better Results!
Queuing Prediction, 80% Utilization
k ,....,, 21
1
2
3
4
……
… ……
… ……
…
13
14
15
16
Fit with MAP(16)
A grid of joint moments and a sequence of autocorrelations fitted, E[XiXi+kXi+k+h]
100 101 102 103 104 105 106 10710-4
10-3
10-2
10-1
100
Pr[Q
ueue
Len
gth
> X]
X [log]
Trace /M/1MAP(16)/M/1
Seagate Web Server Trace
![Page 8: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/8.jpg)
8
Higher Order Correlations V.S. Moments Correlations capture sequence in the time series Correlations are very important
Summary: Matching up to the first three momentsfirst three moments is sufficient Matching higher order correlationshigher order correlations with priority
Fitting Guidelines
Ref: "KPC-Toolbox: Simple Yet Effective Trace Fitting Using Markovian Arrival Processes", G. Casale, E.Z. Zhang, E. Smirni, to appear in QEST’08
![Page 9: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/9.jpg)
9
Challenge (1): Determine MAP Size
Definition: lag-k ACF coefficientk ACF coefficient
MAP(n) Property: Linear Recursive Relationship
of nn consecutive ACF coeffs
BIC Size Selection: Linear regression model on
estimated ACF coeffs BIC value assesses goodness of
model size
kMAP Size Selection - Seagate Trace
- 111600
- 111200
- 110800
- 110400
- 110000
0 10 20 30 40 50 60 70
MAP Size
BIC
MAP(8)
MAP(16) MAP(32)
![Page 10: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/10.jpg)
10
Challenge (2): Trace Descriptor Matching Kronecker Product Composition (KPC)
KPC Properties: Composition of Statistics Moments are composed from moments of small MAPs
MAP Parameterization by KPC to Match Mean and SCV Exactly Higher order correlations as Close as Possible
![Page 11: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/11.jpg)
11
KPC Tool Overview
TraceExtract Statistics
MomentsACF
Correlations…… Size
Selection
MAP(2) MAP(2) MAP(2) MAP(2)……
J = log2N MAP(2)s
MAP(N)MAP(N)
Size of MAPN
Optimization
KPC
This work is supported by NSF grants ITR-0428330 and CNS-0720699
![Page 12: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/12.jpg)
12
Thank you!
![Page 13: KPC-Toolbox Demonstration](https://reader036.fdocuments.in/reader036/viewer/2022062222/56815c6c550346895dca7f6c/html5/thumbnails/13.jpg)
13
What are higher order correlations?
Joint moments of a sequence of inter-arrival times in the time series
Which higher order correlations to fit in KPC? E[XiXi+jXi+j+k], where i can be arbitrary without loss of
generality, and [j,k] chose from a grid of values E.g., [10 100 1000 10000] × [10 100 1000 10000]
= {[10,10], [10,100], [10,10000], …}
Appendix