Lean Analytics Overview

Webinar

Topics

• Set the stage: What is Lean Startup?

• So – What’s Lean Analytics?

• Lean Analytics Concepts

• Innovation Accounting

• Metrics

2

“Lean” Concepts

• Relentlessly eliminate anything that isn’t adding value

• Eliminate time spent on what “we know” we’ll need in

future

• Eliminate inefficient ways of working

• Optimize the whole system

• People doing the work know best how to do it

• Mapping processes and improving

• WoMBaT: Waste of Money, Brains, and Time

Lean Start-Up takes a lean thinking approach

to developing new products. Lean.org

3

What is a Startup?

Start-Ups: Human institution

designed to create new

products and services under

conditions of extreme

uncertainty

• Catalyst that transforms ideas into products

• Includes Entrepreneurs and Intrapreneurs

• Examples: – New Innovation

– Scientific discoveries

– Repurposing existing technology for new use

– New business model

– Product/Service to new location

– Address underserved set of customers

– New internal service

Eric Ries, The Lean Startup

Start-Up: A startup is an

organization formed to search

for a scalable and repeatable

business model.

Bob Dorf, The Startup

Owner’s Manual

4

What is Lean Startup?

• Application of Lean thinking to

the process of innovation

• Adapts Lean ideas in context

of entrepreneurship

• Principled approach to new

product development

• Guidance on how to make

trade-off decisions

• Focused on Validated

Learning

5

Lean Startup Principles

1. Entrepreneurs are everywhere

2. Entrepreneurship is management

3. Product Success depends on learning

4. Leverage Build-Measure-Learn Cycle

5. Measure Learning

Lean Start-Up is a set of practices to help

entrepreneurs increase their odds of building a

successful Product.

6

What Lean Start-Up Is Not --

• Not your traditional way to

create new products

• Not a collection of individual

tactics

• Not a rigid, lockstep process

• Not a lack of discipline

• Not a software development

methodology

• Silver Bullet – Doesn’t fix

everything

7

So – What’s Lean Analytics?

• Advanced work in Lean Startup

• Focused on “Measure” phase of

Build-Measure-Learn Cycle

• Measure and Analyze to:

– Validate problem

– Find the right customer

– Decide what to build

– How to monetize it

– How to spread word

• Core Idea:

– Type business

– Analytics Stage

– Optimize One Metrics That Matters

8

Types of Innovation

• Sustaining Innovation

– Improvements to existing product

– Serving existing customers

– Most companies are good but…

• Disruptive Innovation

– Breakthrough new products

– New sustainable sources of growth

– Companies struggle

9

Lean Startup Overview

Vision

Strategy

Business Plan

Assumptions

Learnings

Product Iterations

Growth

Experiments Sustain

ability?

10

Lean Analytics Stages

Empathy

• Are you solving a problem people care about and will pay for?

• Identify a real problem and real solution

• Get out of the Building, Interviews, Surveys

Stickiness

• Will the dogs eat the dog food?

• Leverage the solution with a small, friendly audience

• Test before going after the masses

Virality

• Will people spread the word?

• Acquisition, Onboarding processes

• Force multiplier for paid promotion

Revenue

• Will they open their pocketbooks?

• Monetize Product, Can you make money?

• Focus on maximizing and optimizing revenue

Scale

• Can we grow the market with sustainability?

• Acquire customers, expand verticals and geographies

• Channels, ecosystem, sustainability

11

Stage Example – Restaurant

Empathy

• Owner learns about diners in area

• Desires, trends, foods not available

• Gets out of building and talks to diners

Stickiness

• Develops menu

• Lots of tests on customers, frequent changes

• High costs, variation, uncertain inventory, giveaways

Virality

• Starts loyalty programs to entice return customers

• Encourage customers to share with friends, friend coupons

• Leverages social media

Revenue

• Work on margins

• Fewer free meals, tighter cost controls

• More standardization

Scale

• Proven sustainability

• Spends on marketing and promotion from revenues, broader advertising

• Launches second restaurant

12

Sample Business Models • E-commerce: Sell things to

customers e.g. Amazon

• SaaS: Software on demand e.g. Salesforce

• Free Mobile App: In-game content drives revenue e.g. Angry Birds

• Media: You create content, make money from advertising e.g. Google’s Search Engine, cNet Home Page

• User-Generated Content: Users create content e.g. Facebook, Wikipedia

• Two-sided: Buyers and sellers come together e.g. eBay, Dating Sites

13

Lean Analytics Concepts

14

Build-Measure-Learn Feedback Loop

Eric Ries, The Lean Startup

• Minimum Viable Product (MVP)

• Fastest full turn of cycle

• Minimum amount of effort

• Minimum development

• Start learning as quickly as possible

• Answer product design or technical questions

• Test fundamental hypotheses

• Usually overestimate needs for MVP

• Are development efforts leading to

real progress

• “Metrics are people, too”

– Represents breathing, thinking,

buying individuals

– Behavior is measureable and

changeable

• Are you making your product better?

How do you know?

• Vital function is learning

• Learn truth of what works in

strategy

• What customers really want

(vs. what they say/think they

want)

• Are we on the path to a

sustainable business?

15

Lean Analytics Cycle

Alistar Croll, Lean Analytics

16

Experiments to Test Strategy

• Leap-of Faith Assumptions: – Riskiest elements of startup’s plan

• Identify risks and assumptions before building anything

• Test those assumptions experimentally

• Devise experiments: – Learn to move numbers closer to those

expected in business model

• Products are an experiment,

MVP is a process

Once clear of leap-of-faith assumptions,

enter Build phase quickly with a

Minimum Viable Product (MVP).

17

Minimum Viable Product

• Fastest full turn of Build-Measure-Learn cycle

• Minimum amount of effort, Minimum development

• Challenges traditional notions of quality

– MVPs sometimes considered low quality by

customers

• Start learning as quickly as possible

– Any work beyond is waste

– Learn what attributes customers care about

• Answer product design or technical questions

• Test fundamental assumptions/hypotheses

• Usually overestimate needs for MVP

18

Early Adopters

• Visionary early customers

• First to adopt

• More forgiving of mistakes

• Expect, even prefer, an 80% solution

• Suspicious of anything too polished

• Features/polish might be a waste of energy for them

19

Validated Learning

• Demonstrated by positive improvements in core metrics

• Synthesis between vision and what customers will accept

• Startup Productivity Metric: – Not – How much stuff we built

– But – How much validated learning we’re getting

• Validate Assumptions (hypotheses)

• Validate Changes

20

Innovation Accounting

Leanstack.com

21

Innovation Accounting

• Quantitative approach to view

results of tuning product

• Geared towards measuring

disruptive innovation

• Answers questions like:

– Are we actually achieving validated

learning?

– Are we learning how to grow a

sustainable business?

22

Pivot or Persevere Decisions

• Does progress show our strategic hypothesis is correct?

• Do we need to make a major change?

• Pivot: structured course correction designed to test a

new fundamental hypothesis

– Product, strategy, growth engine

– Keep one foot rooted in learnings so far

• Persevere: Maintain current path

– Misguided decisions to persevere can destroy creative potential

– Product neither growing nor dying, consuming resources

23

Choosing the Right Metrics

• Qualitative versus Quantitative

• Vanity versus Actionable

• Leading versus Lagging

• Correlated versus Casual

Key Performance Indicators:

Specific metrics that drive

the business

24

Qualitative vs Quantitative Metrics

• Qualitative

– Messy, subjective, imprecise

– “Why”

– Not easily measured

• Quantitative:

– Easy to understand

– “What” and “How Much”

– Ranked, counted, or put on a scale

– Aggregate, extrapolate, put in a spreadsheet

25

Vanity vs Actionable • Vanity:

– Often at a gross up level,

makes things look good

– Ex: Total Signups

• Actionable:

– Analyze customer behavior

– What will I do differently

based on this metric?

– Ex: Percent of users who are

active

26

8 Vanity Metrics to Watch Out For

1. Number of Hits

2. Number of Page Views

3. Number of Visits

4. Number of Unique Visitors

5. Number of Followers,

Friends, Likes

6. Time on Site/Number of

Pages

7. Emails Collected

8. Number of Downloads

27

Cohort Analysis

• Instead of cumulative totals, look at each group

of customers

• Break down these groups

• A/B Testing

• Segmentation

28

Growth Hacking

• Data-driven guerilla marketing

• General technique:

– Find early lifecycle metric as Leading

Indicator (e.g. # friends invited)

– Understand how that metric correlates

to business goal (e.g. driving long-term

engagement)

– Build predictions based on current

Leading Indicator (e.g. engaged users

in 90 days)

– Modify User Experience today to

improve that outcome (e.g. suggest

friends)

29

Leading vs Lagging Metrics

• Leading Metric/Indicator – Tries to predict future

– Relate to early engagement activities

– Tied to business model

– e.g. qualified customers in sales funnel

• Lagging Metric – After the fact

– Indicates that you had a problem

– e.g. Churn

30

Leading & Lagging Indicators

Sales Funnel:

• Sales Team

Leading

indicator,

potential new

customers

P&L:

• Stockholder

Lagging

indicator on

company

performance

Revenue Earned:

• Finance Lagging

indicator, revenue

• Stockholders

Leading indicator

gains/losses

Contracts Signed:

• Sales Team

Lagging Metric,

deals closed

• Finance Leading

indicator, potential

revenue

31

Correlated vs Causal Metrics

• Correlate

– Help predict what will happen

• Causal

– Cause of something happening

– Prove Causality thru Correlation

• A Correlated, Leading Indicator can

predict the future

• A Causal, Leading Indicator can change

the future

32

Correlated vs Causal Metrics

Purewater gazette.net

Correlation is good.

Causality is great!

33

Summary

34

Lean Startup Overview

Vision

Strategy

Business Model

Assumptions

Learnings

Product Iterations

Growth

Experiments Sustain

ability?

35

Analytics Lessons Learned

• Accounting takes on a different role in the uncertain world of Startups

• Know your customer

• Numbers can’t explain everything

• Get out of the building

• Moving targets

• Define what success looks like then: experiment, experiment, experiment

• Use quantitative data to “what” and “how much”

• Use qualitative data to understand “why”

36

Thank You!!! I appreciate your time.

Robert Tyson

Agile Coach & Instructor

37

Lean Startup References

The Lean Startup

by Eric Ries

Lean Analytics

by Alistair Croll & Benjamin Yoskowitz

38

4 Kinds of Information

Alistar Crool, B.Yoskovitz,

Lean Analytics

39