Lean Startup Cost and Benefits

The proof that lean startup saves time and money

Dr. Max Völkel, 2016

2

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Lean Startup Method■You make a plan■Be honest, your plan is just a set of guesses

and hopes we call them assumptions■Try to validate your assumptions by running

cheap experiments, which collect hard data.

■So you build experiments, measure the outcome, and learn, how you need to adapt the plan (“build-measure-learn”)

3

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Assumptions A■A business model can be represented

as 9 building blocks in the Business Model Canvas (BMC), from Alex Osterwalder. 9 assumptions

■Add some assumptions about competition, your team, your sales numbers, market size and you’ll easily have 30-50 assumptions.

4

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Probability p(A)■ An assumption might

be true, or it might not. Let’s say an assumption A is true with probability p(A).

p(A)100%

50%

0%

If we have no information whether A is true or not, p(A)=50% (fifty-fifty).

If p(A) is < 50%, we have the weird situation that our plan is based on an assumption, which we ourselves already think is rather not true.

If p(A)=70%, there is a 30% risk of A being false.

5

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Cost(A)■ If A turns out to be wrong, you need to “adapt your plan”.

That can be finding another customer segment, choosing a new (hopefully better) channel, tweaking the product, … in all cases, you need to do extra work.

■ Extra work for “fixing” your risky project has a cost. ■ Cost can be quantified in Dollars or Euros, or in person-

work-days. Choose a unit and stick with it. Let’s measure in work days. And we really mean time spent working, not waiting.

■ So every failed assumptions has a cost(A), which is really the cost that happens only if A is not true.

6

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Ecost(A)■Now we turn to statistics. We need a concept

called expected value. Example: The expected value of a dice showing

numbers 1,2,3,4,5,6 is 3.5. ■Ecost(A) = (1-p(A)) * Cost

If an assumption A has p(A)=50% and cost(A)=80 days, Ecost(A)=40 days.• If we would do the same risky project again and again, on average we would spend 40 days

doing extra work due to failed hypotheses. Of course, in reality, we only do a startup once, but as we have 30 or more assumptions, they average out and we will spend, on average the expected value of cost.

■Ecost is really just the expected value of cost.

COST AND BENEFIT WITHOUT LEAN STARTUP

7

8

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Without Lean Startup■We have 30 Assumptions

Yes, could be any number, but a range 30-50 is common

■We estimated for each assumption the probability of being true, and the cost of extra work, if the assumption is wrong.

■During out project, some assumptions will turn out to be wrong. We just don’t know which.

■At the end of the project, we will have spent approximately the sum of all Ecost values for doing extra work.

9

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Example with only 5 Assumptions

Assumption A p(A) Cost(A) in days

1-p(A) Ecost(A)=(1-p(A))*Cost

Assumption 1 50% 20 0,50 10,00

Assumption 2 70% 5 0,30 1,50

Assumption 3 66% 5 0,33 1,65

Assumption 4 80% 1 0,20 0,20

Assumption 5 50% 8 0,50 4,00

Sum 39 17,35

Everything else: CalculatedGuessed

10

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Example with only 5 Assumptions

Assumption A p(A)

Assumption 1 50%

Assumption 2 70%

Assumption 3 66%

Assumption 4 80%

Assumption 5 50%

■If all assumptions are right, nothing goes wrong, we have 0 extra days of work – very unlikely.

• Probability of nothing going wrong for our example is 0,5*0,7*0,66*0,8*0,5 = 9,24%. This number if much smaller for more assumptions.

■If absolutely everything goes wrong (Murphys Law), we have 39 days of extra work – very unlikely.

■We should expect extra work of 17,35 days = sum of Ecost values.

COST AND BENEFIT WITH LEAN STARTUP

11

12

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Experiments are costly, too■Right, so let’s call doing experiments validation and let

Cost_V be the cost of doing an experiment (planning, doing, evaluating data, learning).

■The whole purpose of the experiment is to move our vague estimated probability p(A) closer to 100%.

■Let’s accept that we don’t always achieve that. Maybe we had p(A)=60% and after talking to 100 potential

customer we now believe A is true with p = 90%. ■Let’s call the probability after the validation p_V(A).

Note that even though p_V(A) > p(A), the cost(A) remains the same. If things go wrong, we need to do the same amount of work, no matter how sure we have been, that a particular assumption would be true.

13

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Is the experiment worth it?■The expected cost Ecost_V(A) after running an

experiment for validating assumption A isEcost_V(A) = (1-p_V(A)) * Cost + Cost_V Note how we add to cost of running the experiment

(Cost_V) to the expected costs.■The benefit of running the experiment is

Ecost(A) - Ecost_V(A) On average, we should spend less time on extra work

now, because we have a plan that is based more on true facts and less on wishful thinking or naïve assumptions. Of course, if an experiment shows an assumption is wrong, we remove the assumption from our plan and adapt the plan.

14

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Test every assumption?Assumption A p(A) Cost(A)

in days1-p(A) Ecost(A) Cost_V p_V(A) Ecost_V(A)

Assumption 1 50% 20 0,50 10,00 4 95% (1-0,95)*20+4=5,0

Assumption 2 70% 5 0,30 1,50 1 80% (1-0,80)*5+1=2,0

Assumption 3 66% 5 0,33 1,65 6 80% (1-0,80)*5+6=7,0

Assumption 4 80% 1 0,20 0,20 2 90% (1-0,9)*1+2=2,1

Assumption 5 50% 8 0,50 4,00 2 90% (1-0,9)*8+2=2,8

Sum 39 17,35 18,90

Everything else: CalculatedGuessedGuessed

15

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

A Shock!■As you saw on the last slide, the expected

cost without doing any experiments was 17,35 days, and with doing the best experiment we could imagine for each assumption, our sum of expected costs is 18,9 days. Higher! So Lean Startup is just causing extra work?

■We should certainly not run all possible experiments.

■Can we do better?

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Do only some ExperimentsAssumption A p(A) Cost(A)

in days1-p(A) Ecost(A) Cost_V p_V(A) Ecost_V(A)

Assumption 1 50% 20 0,50 10,00 4 95% (1-0,95)*20+4=5,0

Assumption 2 70% 5 0,30 1,50 1 80% (1-0,80)*5+1=2,0

Assumption 3 66% 5 0,33 1,65 6 80% (1-0,80)*5+6=7,0

Assumption 4 80% 1 0,20 0,20 2 90% (1-0,9)*1+2=2,1

Assumption 5 50% 8 0,50 4,00 2 90% (1-0,9)*8+2=2,8

■ For A2, A3 and A4 we should not run the experiments we came up with.

■ Only our experiments for A1 and A5 improve the expected costs. In total, we expect 7,8 days of work instead of 14. So we gain 6,2 days, just for being smart.

17

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Summary■Lean Startup is about doing experiments

which reduce your total expected cost.■You invest extra work (the experiments) and

in return you get either a) a plan based on true assumptions, with data

to proof it (which in turn can convince partners and investors) – or –

b) a new plan, because you found out early where your plan was based on false assumptions before you went ahead and spent a lot of time and money building the product.

18

CO

PYRIG

HT D

R. MA

X VÖ

LKEL., 2015

Thank You■Version 1 published on 2016-02-15■Copyright 2016 by Max Völkel,

www.xam.de ■Licensed under Creative Commons

CC BY-NC-SA 3.0, see https://creativecommons.org/licenses/by-nc-sa/3.0/