Progress for Children: Beyond Averages - Learning from the MDGs
Beyond Averages - Web Performance Meetup
62
BEYOND AVERAGES Dan Kuebrich / @dankosaur
-
Upload
dan-kuebrich -
Category
Technology
-
view
224 -
download
3
description
When raw data becomes overwhelming, we turn to abstraction to understand our world. In examining the performance of our systems, the data is always overwhelming. Solutions like summary statistics have come to our rescue, and they are good—up to a point. In order to truly understand our systems, we need to know when and how to sidestep those abstractions,to get deep, detailed performance insight. At this meetup, I’ll explore techniques for visualizing the underlying structure of performance data and how this empowers drilling down to populations and individual samples in the data set.
Transcript of Beyond Averages - Web Performance Meetup
BEYOND AVERAGESDan Kuebrich / @dankosaur
Beyond Averages
• Abstraction: summary statistics for performance data
• Why performance data is hard
• Visualizing data distributions
A few of my favorite abstractions
A few of my favorite abstractions• Abstraction lets us trade information for
actionability
A few of my favorite abstractions• Abstraction lets us trade information for
actionability
• Min, max, average (“mean”), quantiles, stdev
A few of my favorite abstractions• Abstraction lets us trade information for
actionability
• Min, max, average (“mean”), quantiles, stdev
• That’s a great trade! • ... right?
Averages: average at best
Averages: average at best
Averages: average at best
Averages: average at best
Averages vs Percentiles
[1, 16, 17, 19, 13, 5, 20, 3, 10, 14, 8]
Averages vs Percentiles
[1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]
Averages vs Percentiles
[1, 3, 5, 8, 10, 13, 14, 16, 17, 19, 20]
Average (Mean): 11.54
Median (50th Percentile): 13 90th Percentile: 19
Percentiles: 1 of 100 slices95%
X
Percentiles: 2 of 100 slices95%
10%
X
Y
Percentiles: 2 of 100 slices95%
10%
Percentiles: 2 of 100 slices95%
10%
Percentiles: 2 of 100 slices95%
10%
Computers are hard• Rarely do we have a single distribution underlying the
data
• Different users, different requests, different resources, different instances, different times
Percentiles vs Distributions
http://en.wikipedia.org/wiki/Percentile
Percentiles vs Distributions
[13, 13, 13, 13, 13, 13, 19, 19, 19, 19, 19]
Median (50th Percentile): 13 90th Percentile: 19
Rarely do we have a single normal distribution underlying the data
Median Mean 90th
Median Mean 90th
The Log-Normal Distribution
(source: http://www.geo.mtu.edu/volcanoes/vc_web/background/S_chem.html)
Log-Normal Distribution
(source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)
Log-Normal Distribution
(source: http://en.wikipedia.org/wiki/File:Comparison_mean_median_mode.svg)
Log-Normal Distribution
Is there a place between Averageland and “A Beautiful Mind”?
http://now-here-this.timeout.com/2012/10/07/crazy-walls-of-clues-from-tv-film-reviewed-by-carrie-from-homeland/
Histograms
Freq
uenc
y (e
g. #
of c
alls
)
Value (eg. latency)
Populations revisited95%
10%
HistogramsFr
eque
ncy
(eg.
# o
f cal
ls)
Value (eg. latency)
Populations re-revisited95%
10%?
3d Histograms?
Freq
uenc
y (e
g. #
of c
alls
)
Value (eg. latency)
3d Histograms?Fr
eque
ncy
(eg.
# o
f cal
ls)
Value (eg. latency)
Time
Heatmaps
Freq
uenc
y (e
g. #
of c
alls
)
Value (eg. latency)
HeatmapsFr
eque
ncy
(eg.
# o
f cal
ls)
Value (eg. latency)
HeatmapsFr
eque
ncy
(eg.
# o
f cal
ls)
Value (eg. latency)
HeatmapsVa
lue
(eg.
late
ncy)
Time
HeatmapsVa
lue
(eg.
late
ncy)
Time
Latency in the wild…
http://sciencefiction.com/2013/10/24/throwback-thursday-jurassic-park/
Average, or Absolute?
Average, or Absolute?
Multi-modal Data
Multi-modal Data
Multi-modal Drill-down
Multi-modal Drill-down
Multi-modal Drill-down
Long Tails and Outliers
bottom 98%
Long Tails and Outliers
bottom 98%
Long Tails and Outliers
all of it
Long Tails and Outliers
Added Population
Added Population
Added Population
Thanks!
Sign up for a free TraceView account at appneta.com/products/traceview-free-account
