Startupfest 2015: ALISTAIR CROLL (Solve for Interesting) - Lightning Keynote
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
-
Upload
lean-analytics -
Category
Business
-
view
10.076 -
download
2
Transcript of Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Percent of businesses that fail
Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10
71%69%66%63%60%55%50%44%36%25%
http://www.statisticbrain.com/startup-failure-by-industry/
Still operating after 4 yearsFinance Insurance and Real Estate
Education and HealthAgriculture
ServicesWholesale
MiningManufacturing
ConstructionRetail
Transportation Communication and UtilitiesInformation 37%
45%
47%
47%
49%
51%
54%
55%
56%
56%
58%
http://www.statisticbrain.com/startup-failure-by-industry/
Everyone’s idea is the best right?
People love this part!
(but that’s not always a good thing)
This is where things fall apart.
No data, no learning.
Most startups don’t know what they’ll be when they grow up.
Hotmailwas a database company
Flickrwas going to be an MMO
Twitter was a podcasting company
Autodesk made desktop automation
Paypalfirst built for Palmpilots
Freshbookswas invoicing for a web design firm
Wikipedia was to be written by experts only
Mitelwas a lawnmower company
In a startup, the purpose of analytics is to iterate to product/market fit
before the money runs out.
A good metric is:
Understandable
If you’re busy explaining the data, you won’t be busy acting on it.
Comparative
Comparison is context.
A ratio or rate
The only way to measure change and roll up the tension between two metrics (MPH)
Behaviorchanging
What will you do differently based on the results you collect?
The simplest rule
badmetric.
If a metric won’t change how you behave, it’s a
h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself.
Acquisition
Hybrid
Loyalty
70%of retailers
20%of retailers
10%of retailers
You are just like
Customers that buy >1x in 90d
Once
2-2.5per year
>2.5per year
Your customers will buy from you
Then you are in this mode
1-15%
15-30%
>30%
Low acquisition cost, high checkout
Increasing return rates, market share
Loyalty, selection, inventory size
Focus on
(Thanks to Kevin Hillstrom for this.)
MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activ
e use
rs
Cohort: Comparison of similar groups along a timeline. (this is the April cohort)
A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.)
Multivariateanalysis Changing several things at once to see which correlates with a result.
☀☁☀☁
Segment: Cross-sectional
comparison of all people divided by
some attribute (age, gender, etc.)
☀
☁
January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50Is this company growing or stagnating?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
How about this one?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
Averages $7 $5 $3 $1 $0.5
Look at the same data in cohorts
Eric’s three engines of growth
Virality
Make people invite friends.
How many they tell, how fast they
tell them.
Price
Spend money to get customers.
Customers are worth more than
they cost.
Stickiness
Keep people coming back.
Approach
Get customers faster than you
lose them.
Math that matters
Dave’s Pirate MetricsAARRR
AcquisitionHow do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
ActivationDo drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
RetentionDoes a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
RevenueDo you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
ReferralDo users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...
Stage
EMPATHY I’ve found a real, poorly-met need that a reachable market faces.
STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for.
VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives.
REVENUE The users and features fuel growth organically and artificially.
SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem.
GateTh
e fiv
e st
ages
Six business model archetypes.
E-commerce SaaS MediaMobileapp
User-gencontent
2-sidedmarket
The business you’re in
(Which means eye charts like these.)
Customer Acquisition Cost
paid direct search wom inherent virality
VISITOR
Freemium/trial offer
Enrollment
User
Disengaged User
Cancel
Freemium churn
Engaged User
Free user disengagement
Reactivate
Cancel
Trial abandonment rate
Invite Others
Paying Customer
Reactivationrate
Paid conversion
FORMER USERS
User Lifetime Value
Reactivate
FORMER CUSTOMERS
Customer Lifetime Value
Viral coefficientViral rate
Resolution
Support data
Account Cancelled Billing Info Exp.
Paid Churn Rate
Tiering
Capacity Limit
Upselling rate Upselling
Disengaged DissatisfiedTrial Over
Model + Stage = One Metric That Matters.
One Metric That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media UCG
Empathy
Stickiness
Virality
Revenue
ScaleThe
stag
e yo
u’re
at
Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/
Empathy
Stickiness
Virality
Revenue
Scale
E-commerce SaaS MediaMobile
appUser-gencontent
2-sidedmarket
Interviews; qualitative results; quantitative scoring; surveys
Loyalty, conversion
CAC, shares, reactivation
Transaction, CLV
Affiliates, white-label
Engagement, churn
Inherent virality, CAC
Upselling, CAC, CLV
API, magic #, mktplace
Content, spam
Invites, sharing
Ads, donations
Analytics, user data
Inventory, listings
SEM, sharing
Transactions, commission
Other verticals
(Money from transactions)
Downloads, churn, virality
WoM, app ratings, CAC
CLV, ARPDAU
Spinoffs, publishers
(Money from active users)
Traffic, visits, returns
Content virality, SEM
CPE, affiliate %, eyeballs
Syndication, licenses
(Money from ad clicks)
Baseline: 5-7% growth a week
“A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users.
Paul Graham, Y Combinator
• Are there enough people who really care enough to sustain a 5% growth rate?
• Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution
• Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week
• Once you’re generating revenues, they should grow at 5% a week
It’s oxygenYou need customers to keep learning It’s a substitute for solvency
Photo by Paul Miller on Flickr. https://w
ww
.flickr.com/photos/94674772@
N03/8788576498
Baseline: 10% visitor engagement/day
Fred Wilson’s social ratios
30% of users/month use web or mobile app
10% of users/day use web or mobile app
1% of users/day use it concurrently
Baseline: 2-5% monthly churn• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.• Get below a 5% monthly churn rate before you know you’ve got a
business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok)
• Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
Baseline: Calculating customer lifetime
25%monthly churn
100/25=4The average
customer lasts 4 months
5%monthly churn
100/5=20The average
customer lasts 20 months
2%monthly churn
100/2=50The average
customer lasts 50 months
Baseline: CAC under 1/3 of CLV• CLV is wrong. CAC Is probably wrong, too.• Time kills all plans: It’ll take a long time to find
out whether your churn and revenue projections are right
• Cashflow: You’re basically “loaning” the customer money between acquisition and CLV.
• It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner.
Lifetime of 20 mo.$30/mo. per
customer$600 CLV
$200 CACNow segment those users!
1/3 spend
Draw a new linePivot orgive up
Try again
Success!
Did we move the needle?
Measure the results
Make changes in production
Design a test
Hypothesis
With data:find a
commonality
Without data: make a good
guess
Find a potential improvement
Draw a linePick a KPI
Gut instinct (hypothesis)Professional photography helps AirBnB’s business
Candidate solution (MVP)20 field photographers posing as employees
Measure the resultsCompare photographed listings to a control group
Make a decision Launch photography as a new feature for all hosts
Draw a new linePivot orgive up
Try again
Success!
Did we move the needle?
Measure the results
Make changes in production
Design a test
Hypothesis
With data:find a
commonality
Without data: make a good
guess
Find a potential improvement
Draw a linePick a KPI
“Gee, those houses that do well look really
nice.”
Maybe it’s the camera.
“Computer: What do all the
highly rented houses have in
common?”
Camera model.
With data:find a commonality
Without data: make a good guess
Landing page design A/B testing
Cohort analysis General analytics
URL shortening
Funnel analytics
Influencer Marketing
Publisher analytics
SaaS analytics
Gaming analytics
User interaction Customer satisfaction KPI dashboardsUser segmentation
User analytics Spying on users
When you’re a startup your goal is to find a sustainable,
repeatable business model.
When you’re a big company your goal is to perpetuate one.
In a startup, the purpose of analytics is to iterate to product/market fit
before the money runs out.
“The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.”
Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
Alistair Croll [email protected] @acroll
Ben Yoskovitz [email protected] @byosko