Lean Startup Analytics/Metrics

28
Vanity Dies Hard Back to Basics

description

Presented at Lean Startup Circle on 4/15/14 for their back to basics night. The theme for the evening was analytics.

Transcript of Lean Startup Analytics/Metrics

Page 1: Lean Startup Analytics/Metrics

Vanity Dies Hard Back  to  Basics  

Page 2: Lean Startup Analytics/Metrics

@ElliotSusel  

Page 3: Lean Startup Analytics/Metrics

Vanity Metrics Actionable Metrics Innovation Accounting Systems

Page 4: Lean Startup Analytics/Metrics

Vanity Metric Low  correla/on  to  what  makes  your  

business  successful  

Page 5: Lean Startup Analytics/Metrics

These are important…

But  let’s  not  overstate  their  significance  

Page 6: Lean Startup Analytics/Metrics

Success Theater Look  at  our  [insert  vanity  metric]!  

Our  business  is  amazing!  

Page 7: Lean Startup Analytics/Metrics

What then DO we measure?

Page 8: Lean Startup Analytics/Metrics

Actionable Metric High  correla/on  to  what  makes  your  

business  successful  

Page 9: Lean Startup Analytics/Metrics

Actionable Metric

Page 10: Lean Startup Analytics/Metrics

But there’s a problem The  ac/onable  metric  is  a  lie  

Page 11: Lean Startup Analytics/Metrics

What impacts ride volume?

Page 12: Lean Startup Analytics/Metrics

•  Outreach  •  Promo/ons  •  Support  •  Technology  

Stuff we control

Page 13: Lean Startup Analytics/Metrics

•  Weather  •  Seasonality  •  Holidays  •  Events  (Sports,  Religious,  Concerts,  Others…)  •  System  Outages  (Fleet,  Dispatch,  Internal)  

 

Stuff we don’t control

Page 14: Lean Startup Analytics/Metrics

THIS IS SCARY We  think  we  have  meaningful  data  We  do  not  have  meaningful  data  

Page 15: Lean Startup Analytics/Metrics

Meaningful Conclusions?

Page 16: Lean Startup Analytics/Metrics

Innovation Accounting Understand  the  factors  impac/ng  

your  metrics  

Page 17: Lean Startup Analytics/Metrics

Understanding Ride Volume

•  New  vs  Return  Customers  •  Distribu/on  by  plaMorm  •  Distribu/on  by  fleet/geographic  region  •  Distribu/on  by  day  of  week  and  /me  of  day  

Page 18: Lean Startup Analytics/Metrics

Why? Validated  learning  is  a  startup’s  

most  valuable  commodity  

Page 19: Lean Startup Analytics/Metrics

Nice concepts How  do  we  execute?  

Page 20: Lean Startup Analytics/Metrics

A SYSTEM, YOU SAY?

Losers  have  goals  Winners  create  systems  

Page 21: Lean Startup Analytics/Metrics

Components of Our System  

•  Every  development  request  has  a  goal  •  Every  goal  has  quan/fica/on  •  Architecture  review  includes  analy/cs  •  Code  review  includes  analy/cs  •  Every  PROD  deployment  requires  hooks  to  monitoring/analy/cs  tools  

Page 22: Lean Startup Analytics/Metrics

Stuff we’ve learned Or  at  least,  closing  thoughts  

Page 23: Lean Startup Analytics/Metrics

Culture Matters If  leadership  doesn’t  value  analy/cs,  

no  one  will  value  analy/cs  

Page 24: Lean Startup Analytics/Metrics

Over-reporting Don’t  do  it  

Page 25: Lean Startup Analytics/Metrics

Accidental Learning Informa/on  radiators  are  your  friend  

Page 26: Lean Startup Analytics/Metrics

Create a System A  project  accounts  for  metrics  from  

incep/on  to  delivery  

Page 27: Lean Startup Analytics/Metrics

Then tune it Because  you’re  never  done  learning  

Page 28: Lean Startup Analytics/Metrics

Questions? @ElliotSusel