Monitorama14: A Melange of Methods for Manipulating Monitored Data

download Monitorama14: A Melange of Methods for Manipulating Monitored Data

of 73

  • date post

    18-Dec-2014
  • Category

    Technology

  • view

    2.194
  • download

    2

Embed Size (px)

description

Discusses The Greatest Scatter Plot (Hubble 1929), Irregular Time Series (Harmonic Mean), Zipf’s Law of Words, Oracle Query Times, and Eleventh Hour Spikes.

Transcript of Monitorama14: A Melange of Methods for Manipulating Monitored Data

  • 1. A Melange of Methods for Manipulating Monitored Data Converging on Consistency Neil Gunther @DrQz en.wikipedia.org/wiki/Neil_J._Gunther Performance Dynamics Monitorama PDX May 6, 2014 SM c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 1 / 52
  • 2. Introductions c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 2 / 52
  • 3. c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 3 / 52
  • 4. I didnt do Monitorama Berlin c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 3 / 52
  • 5. I didnt do Monitorama Berlin I didnt get the memo about plane crashes c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 3 / 52
  • 6. I didnt do Monitorama Berlin I didnt get the memo about plane crashes Sorry... Deal with it c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 3 / 52
  • 7. I didnt do Monitorama Berlin I didnt get the memo about plane crashes Sorry... Deal with it SFO runway 28L, 11:28 a.m., July 6, 2013 c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 3 / 52
  • 8. I didnt do Monitorama Berlin I didnt get the memo about plane crashes Sorry... Deal with it SFO runway 28L, 11:28 a.m., July 6, 2013 Asiana Airlines Flight 214 landing arse-backwards c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 3 / 52
  • 9. I didnt do Monitorama Berlin I didnt get the memo about plane crashes Sorry... Deal with it SFO runway 28L, 11:28 a.m., July 6, 2013 Asiana Airlines Flight 214 landing arse-backwards (sans tail) c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 3 / 52
  • 10. Asiana pilots appear to be overly reliant on instrument-guided landings and lack the training to touch down manually. SFO Commissioner Eleanor Johns c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 4 / 52
  • 11. A Message from Your Sponsors Dont be too reliant on your instruments (strip charts, colored dials, shiny things) c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 5 / 52
  • 12. Consistency 1 Its not about pretty pictures c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 6 / 52
  • 13. Consistency 1 Its not about pretty pictures 2 Its not about whiz bang tools c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 6 / 52
  • 14. Consistency 1 Its not about pretty pictures 2 Its not about whiz bang tools 3 Its not about fancy math c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 6 / 52
  • 15. Consistency 1 Its not about pretty pictures 2 Its not about whiz bang tools 3 Its not about fancy math 4 Data are usually trying to tell you something c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 6 / 52
  • 16. Consistency 1 Its not about pretty pictures 2 Its not about whiz bang tools 3 Its not about fancy math 4 Data are usually trying to tell you something 5 Your interpretation has to be consistent with other data c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 6 / 52
  • 17. Consistency 1 Its not about pretty pictures 2 Its not about whiz bang tools 3 Its not about fancy math 4 Data are usually trying to tell you something 5 Your interpretation has to be consistent with other data 6 Your interpretation has to be consistent with other information c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 6 / 52
  • 18. Consistency 1 Its not about pretty pictures 2 Its not about whiz bang tools 3 Its not about fancy math 4 Data are usually trying to tell you something 5 Your interpretation has to be consistent with other data 6 Your interpretation has to be consistent with other information This talk is about Converging on consistency by example c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 6 / 52
  • 19. The Greatest Scatter Plot Topics 1 The Greatest Scatter Plot 2 Irregular Time Series 3 The Power of Power Laws Zipfs Law of Words Database Query Times Eleventh Hour Spikes c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 7 / 52
  • 20. The Greatest Scatter Plot The Greatest Scatter Plot c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 8 / 52
  • 21. The Greatest Scatter Plot Goggle up! Science ahead... c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 9 / 52
  • 22. The Greatest Scatter Plot Some Monitored Data 5 10 15 20 0.00.51.01.52.0 Time Metric1 5 10 15 20 -2002006001000 Time Metric2 Two time series, two metrics: Metric 1 and Metric 2 c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 10 / 52
  • 23. The Greatest Scatter Plot Scatter Plot 0.0 0.5 1.0 1.5 2.0 05001000 Metric 1 Metric2 Are Metric 1 and Metric 2 related in any way? c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 11 / 52
  • 24. The Greatest Scatter Plot Linear Regression 0.0 0.5 1.0 1.5 2.0 05001000 Metric 1 Metric2 LSQ t: Metric2 = 423.94 Metric1 and R2 = 0.82 c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 12 / 52
  • 25. The Greatest Scatter Plot This is Not the End This is just the beginning Need to reach consistency 1 Is the linear t still a reasonable choice? 2 What is the meaning of the slope ? 3 Willing to extrapolate this model into the future? c 2014 Performance Dynamics A Melange of Methods for Manipulating Monitored Data May 6, 2014 13 / 52
  • 26. The Greatest Scatter Plot The most important scatter plot in history (1929) le on the expanding universe appeared in PNAS in 1929 [Hubble, E. P. (1929) Proc. Natl. Acad. Sci. USA 15, that a galaxys distance is proportional to its redshift, is so well known and so deeply embedded int