Download - Beyond Averages

Transcript
Page 1: Beyond Averages

BEYOND AVERAGESDan Kuebrich / appneta.com

Page 2: Beyond Averages
Page 3: Beyond Averages

A few of my favorite abstractions

•Abstraction lets us trade information for actionability

•Min, max, average, quantiles, stdev

•That’s a great trade!• ... right?

Page 4: Beyond Averages

Averages: average at best

Page 5: Beyond Averages

Averages: average at best

Page 6: Beyond Averages

Averages: average at best

Page 7: Beyond Averages

Averages: average at best

Page 8: Beyond Averages

Percentiles: 1 of 100 slices

95%

Page 9: Beyond Averages

Percentiles: 2 of 100 slices

95%

10%

Page 10: Beyond Averages

Percentiles: 2 of 100 slices

95%

10%

Page 11: Beyond Averages

Percentiles: 2 of 100 slices

95%

10%

Page 12: Beyond Averages

Percentiles: 2 of 100 slices

95%

10%

Page 13: Beyond Averages

Computers are hard

• Rarely do we have a single normal distribution underlying the data

• Different users, different requests, different resources, different instances, different times

Page 14: Beyond Averages

Is there a place between Averageland and “A Beautiful Mind”?

http://now-here-this.timeout.com/2012/10/07/crazy-walls-of-clues-from-tv-film-reviewed-by-carrie-from-homeland/

Page 15: Beyond Averages

HistogramsFr

eque

ncy

(eg.

# o

f cal

ls)

Value(eg. latency)

Page 16: Beyond Averages

Populations revisited

95%

10%

Page 17: Beyond Averages

HistogramsFr

eque

ncy

(eg.

# o

f cal

ls)

Value(eg. latency)

Page 18: Beyond Averages

Populations re-revisited

95%

10%?

Page 19: Beyond Averages

3d Histograms?Fr

eque

ncy

(eg.

# o

f cal

ls)

Value(eg. latency)

Page 20: Beyond Averages

3d Histograms?Fr

eque

ncy

(eg.

# o

f cal

ls)

Value(eg. latency)

Time

Page 21: Beyond Averages

HeatmapsFr

eque

ncy

(eg.

# o

f cal

ls)

Value(eg. latency)

Page 22: Beyond Averages

HeatmapsFr

eque

ncy

(eg.

# o

f cal

ls)

Value(eg. latency)

Page 23: Beyond Averages

HeatmapsFr

eque

ncy

(eg.

# o

f cal

ls)

Value(eg. latency)

Page 24: Beyond Averages

HeatmapsVa

lue

(eg.

late

ncy)

Time

Page 25: Beyond Averages

OK, but what about the real world?

http://www.justincarmony.com/blog/2012/06/05/customizing-graphite-charts-for-clearer-results/

Page 26: Beyond Averages

Mystery #1

Page 27: Beyond Averages

Mystery #1

Page 28: Beyond Averages

Mystery #1

Page 29: Beyond Averages

Mystery #1

Page 30: Beyond Averages

Mystery #1

Page 31: Beyond Averages

Mystery #2

Page 32: Beyond Averages

Mystery #2

bottom 98%

Page 33: Beyond Averages

Mystery #2

all of it

Page 34: Beyond Averages

Mystery #3

Page 35: Beyond Averages

Mystery #3: UNSOLVED