Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in...
Transcript of Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in...
![Page 1: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/1.jpg)
Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those of Alphabet, Inc or its subsidiaries, including Google, Inc.
Select images and formulae are provided with permission from Google, Inc
Percentile-Based ApproachTo Forecasting Workload
Growth
Alex Gilgur, Douglas Browning, Stephen Gunn,Xiaojun Di, Wei Chen, and Rajesh Krishnaswamy
IT Capacity and Performance 41st International Conference by
the Computer Measurement Group (CMG'15)
San Antonio, TX November 5, 2015Session 525
![Page 2: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/2.jpg)
“What's in a name? That which we call a rose / By any other name /Would smell as sweet”
● Useful Workload Measures:○ Outlier Boundary ○ Mean + “Z” Standard Deviations (e.g. “3 sigma”, “6 sigma”, etc.)○ 95th percentile○ 90th percentile○ 75th percentile○ Simple Average (Mean)○ Median○ 25th percentile
● Define Workload via Little's Law:○ Number of units of work in the system
■ Number of packets in flight■ Number of queries in queue
○ W = X * T
![Page 3: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/3.jpg)
How often do you see such patterns?
How do you predict them?
![Page 4: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/4.jpg)
● Workload Forecasting○ When Classical Methods Fail○ Workload statistics:
■ “Z-sigma”■ Quantile Regression
● p95● Outlier Boundaries
○ Knees, Hyperbolae, and Sensitivity○ Quantile Compression○ Predicting the Workload○ Use Cases
What We Will Talk About
![Page 5: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/5.jpg)
“It may be normal, darling; but I'd rather be natural.”Truman Capote, Breakfast at Tiffany’s
Usual Assumptions:
● Residuals are Normally distributed● Mean and StDev of residuals are constant
![Page 6: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/6.jpg)
“Double, Double, Toil and Trouble”
![Page 7: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/7.jpg)
Solutions
And the Winner Is...
![Page 8: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/8.jpg)
… Predicting 95th Percentile Directly !
![Page 9: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/9.jpg)
Why not Split the Workload into Two Servers?
● Sometimes app1 and app2 have to go to the same server for processing.
● To size the server (VM/Network Link/Storage LUN/… ), we only want to forecast the upper bound.
● Sometimes it’s hard to get additional capacity: ○ budget, justification, approvals, etc.○ Cloud helps, but...
Percentile-based Modeling is often the only solution
![Page 10: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/10.jpg)
QuantReg for the 2 types of workload
fits very nicely in both cases
![Page 11: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/11.jpg)
Should we Size Hardware for 95%-ile of Workload?
5% of the time SLA (99.9%? 99.999%?)
will be violated
![Page 12: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/12.jpg)
IQR vs. 5th and 95th Percentiles
IQR excludes “true” outliers
IQR (Tukey’s method) 5th and 95th Percentiles
![Page 13: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/13.jpg)
Shouldn’t we Size Resources for Non-Outliers Instead?
John Tukey’s IQR method
● Why?○ Normality does not matter○ 5th & 95th percentiles in this scenario
would have “outlawed” a good part of data points that are NOT outliers
● Why Not?○ Multi-Modal distributions
We size for SLA, as long as traffic stays within outlier boundaries
![Page 14: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/14.jpg)
Tukey’s Boundaries in Trended Data
![Page 15: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/15.jpg)
● don’t predict based on all data:○ find natural groupings (GMM, DBSCAN, ...) ○ then fit the model○ use the higher cluster to guarantee QoS
Tukey’s Method● robust boundaries● distribution-agnostic● can be used to guarantee high QoS
○ Unimodal Distribution ○ Multi-Modal Distribution
![Page 16: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/16.jpg)
Long Story Short
● Forecast Percentiles
● Find Natural Groupings
● Size For Outlier Boundaries
![Page 17: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/17.jpg)
● Workload Forecasting○ When Classical Methods Fail○ Workload statistics:
■ “Z-sigma”■ Quantile Regression
● p95● Outlier Boundaries
○ Knees, Hyperbolae, and Sensitivity○ Quantile Compression○ Predicting the Workload○ Use Cases
What We Will Talk About Next
![Page 18: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/18.jpg)
Knees, Hyperbolae, and Sensitivity
Capacity:
Workload:
In a closed (constrained) system, sensitivity of throughput to latency increases with the throughput
![Page 19: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/19.jpg)
In Human Terms
● As throughput increases, latency can only increase.
● As latency increases, throughput in a constrained queueing system can only decrease.
● As we increase throughput in a constrained system near its saturation point, its upper percentiles must grow at a slower pace than lower percentiles.
![Page 20: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/20.jpg)
In Mathematical Terms
Quantile Compression Theorem:
IF raw demand on a constrained system X′ is
moderated via a monotonically increasing damped function X = f(X′),
THEN, as the system is approaching saturation,
smaller percentiles of moderated demand X grow on average faster than higher percentiles.
This is only a presentation; for mathematical proof, please see the paper.
![Page 21: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/21.jpg)
Long Story Short
“It’s just there...”-Miles Davis
Quantile Compression:As the system is approaching saturation, smaller percentiles of moderated demand X grow on average faster than higher percentiles.
![Page 22: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/22.jpg)
In Practical Terms: are We Constrained?
Percentile trajectories diverge; we are NOT constrained here.
![Page 23: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/23.jpg)
In Practical Terms: are We Constrained?
p5 and p95 trajectories converge; we ARE GETTING constrained here.
![Page 24: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/24.jpg)
Percentile trajectories are almost all parallel; we are almost NOT constrained here.
In Practical Terms: are We Constrained?
“It’s always the quiet ones”
![Page 25: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/25.jpg)
In Practical Terms: are We Constrained?Predictions made:p75` > p90` > p50` > p95`
Line Predicted by p90
Line Predicted by p50
Line Predicted by p75
Line Predicted by p95
Expected Growth Rates:p95` > p90` > p75` > p50`
Observed Growth Rates:p75` > p90` > p50` > p95`
p95 trajectory is growing slower than p50; we ARE constrained here.
![Page 26: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/26.jpg)
In Statistical Terms: When Resource is Unconstrained
Unbounded Resource Throughput: Unimodal; Asymmetric; Skew is Constant
![Page 27: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/27.jpg)
Bounded (Constrained) Resource Throughput: may become Bimodal; Skew may vary
In Statistical Terms: When Resource is Constrained
![Page 28: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/28.jpg)
Can we Measure Asymmetry?
![Page 29: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/29.jpg)
Not All Distributions are Easy to Deal With
What if...
...Mean and
Variance are
undefined
?
![Page 30: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/30.jpg)
Percentiles win!
What if...
...Mean and
Variance are
undefined
?
![Page 31: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/31.jpg)
Long Story Short
When resource is constrained:
1. Distribution changes:a. becomes left-skewedb. becomes bimodal
2. Skew is very important
3. Percentile-based Skew is the preferable statistic
![Page 32: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/32.jpg)
Some More Examples that you May Have Seen Before
![Page 33: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/33.jpg)
Growth Was Constrained
![Page 34: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/34.jpg)
Unconstrained Growth
Right-Skewed (long right tail)
![Page 35: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/35.jpg)
Controlled Growth
Left-Skewed (long left tail)
![Page 36: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/36.jpg)
What We Will Talk About Next
● Workload Forecasting○ When Linear Regression fails○ Workload statistics:■ “Z-sigma”■ Quantile Regression● p95● Outlier Boundaries
○ Knees, Hyperbolae, and Sensitivity○ Quantile Compression○ Predicting the Workload○ Use Cases
![Page 37: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/37.jpg)
“None that I know will be, much that I fear may chance”
● Regression: ○ Business Metrics○ Little's Law○ Time-related Covariates
● Time Series Analysis (Forecasting):○ EWMA○ ARIMA
![Page 38: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/38.jpg)
Is it right to Size Resources Using Upper Percentiles of Bounded Data?
Forecasting demand using bounded data leads to undersizing the resource
Doing so is the path to the dark side.
Resource Constraint =>
Quantile Compression =>
Underforecasting the load =>
Undersizing the resource
Quantile Compression:As the system is approaching saturation, smaller percentiles of moderated demand X grow on average faster than higher percentiles.
![Page 39: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/39.jpg)
Can we infer unbounded lines from bounded data?
TimeStamp
Ske
w1. Find Skew for Unbounded Data
2. Forecast Upper and Lower Percentiles to the Time Horizon of Interest
3. Infer Unbounded Upper Percentiles (Skew = const)
4. If (unbounded = forecasted) => system is still unbounded
5. If (unbounded > forecasted & forecast > history) => system will be constrained
![Page 40: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/40.jpg)
Throughput Forecasting Algorithm
Get U(t)StartIdentify the most
appropriate trend type
Done
Predict Trajectories for the LB
(p25) and Median
(LB’, M’) = Prediction for Low
Bounds and Median
Save the forecast
For each timestamp
Build hourly
boxplots
data
Thro
ughput
Thro
ughput
![Page 41: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/41.jpg)
Identify the most
appropriate trend type
Thro
ughput LINT
hro
ughput
LOG
Thro
ughput
EXPThro
ughput
QUAD
Th
roughput
PWRThro
ughput
R2 = 0.45
R2 = 0.34
R2 = 0.47
R2 = 0.38
R2 = 0.46
Trend Type Selection
● we know the variance is huge● we are selecting TREND TYPE● we are NOT selecting MODEL
![Page 42: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/42.jpg)
Now we can use T-test
LINThro
ughput R2 = 0.45
QUAD
Thro
ughput
R2 = 0.46
EXPThro
ughput
R2 = 0.47
LOGThro
ughput
R2 = 0.34
PWRThro
ughput
R2 = 0.38
A few words about R2
![Page 43: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/43.jpg)
Trend Type SelectionT
hro
ughput
LIN
Thro
ughput
R2 = 0.45
● we know the variance is huge● we are selecting TREND TYPE● we are NOT selecting MODEL
MODELS: “LIN”, “PWR”, “EXP”, “LOG”,“QUAD”
Identify the most
appropriate trend type
![Page 44: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/44.jpg)
Long Story Short
Forecasting Algorithm:
1. Compute the Skew
2. Identify the Trend Type
3. Forecast p25 and p50
4. Apply Skew to Compute Upper Percentiles
5. Compute Outlier Boundaries
![Page 45: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/45.jpg)
● Workload Forecasting○ When Classical Methods Fail○ Workload statistics:
■ “Z-sigma”■ Quantile Regression
● p95● Outlier Boundaries
○ Knees, Hyperbolae, and Sensitivity○ Quantile Compression○ Predicting the Workload○ Use Cases
What We Will Talk About Next
![Page 46: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/46.jpg)
Thro
ughput
Th
rou
gh
put
Use Cases: Unbounded: How Far to the Threshold?
Threshold
1000
250
Non-Outliers above threshold
Pr {traffic > threshold} > 5%
![Page 47: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/47.jpg)
Another interesting scenario: Forecasting Resource Congestion Zone
![Page 48: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/48.jpg)
Forecasting ResourceCongestion Zone
By predicting collision points for different percentiles,we can get a general idea of a Resource Congestion Zone
HAL9000: I've just picked up a fault
in the AE35 unit. It's going to go
100% failure in 72 hours.
![Page 49: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/49.jpg)
Use Cases: Unbounded: How Much to Add?
(unbounded = forecasted) => system is still unbounded
![Page 50: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/50.jpg)
Use Cases: Bounded (Congested): How Much to Add?
(unbounded > forecasted) => system was, and will be, constrained
![Page 51: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/51.jpg)
Use Cases: Bounded (Congested): How Much to Add?
(unbounded > forecasted) => system may have been, and will be, constrained
![Page 52: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/52.jpg)
Long Story Short
● Feedback Loop & Quantile Compression:
○ “It’s just there”:■ explicitly, via the protocol.■ implicitly, in the saturation dynamics.
● Do not assume anything!
○ Especially about shapes of distributions.
● Do not forecast p95!
○ Forecast Outlier Boundaries instead.○ Mean and Variance are overrated!
● Do Size Hardware for the would-have-been-
unbounded Forecasts
![Page 54: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/54.jpg)
Appendix
![Page 55: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/55.jpg)
“Big 7” of Linearizable Equations
odds log
![Page 56: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/56.jpg)
Is it Right to Size Resources Using Upper Percentiles?
Quantile Compression:As the system is approaching saturation, smaller percentiles of moderated demand X grow on average faster than higher percentiles.
![Page 57: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/57.jpg)
Is it Right to Size Resources Using Upper Percentiles?
![Page 58: Percentile-Based Approach To Forecasting Workload Growth€¦ · Views and opinions expressed in this presentation are solely of its authors and do not necessarily represent those](https://reader033.fdocuments.in/reader033/viewer/2022050310/5f71d6c192b63a697025e915/html5/thumbnails/58.jpg)
ForecastingMethods:
● EWMA
● ARIMA
● Regression
EWMA models are very specific and computationally fast, but they have to be told trend
(linear or exponential) and seasonality (additive or multiplicative).
ARIMA model will implicitly account for trends, seasonality, and stationarity of the data.
Autocorrelation of ARIMA residuals provide all the periodicities that have been missed.
For stationary data, use ARIMA
For non-stationary data, use EWMA
EWMA and ARIMA overlap
When to use Regression:
● data are monotonic.
● seasonality is NOT statistically significant.
● EWMA and ARIMA fail.
When to use Quantile Regression:
● Upper and Lower bounds behave differently.
● Outliers are possible.
For each data set, we can run a model competition, computing forecast model quality based
on a weighted sum of model goodness of fit, model suitability for forecasting, data stationarity
and data variability, and selecting the model that works best for each data set.
EWMA
ARIMA
Quantile Regression