The Innovator’s Guide to the Scientific Method

Tendayi Viki University of Kent

Website: www.tendayiviki.com Follow Me: @tendayiviki

Pearson Publishing

About Me:

Academic: University of Kent

Several Previous Startups: Tasksauce, Book Editions,

Research Innovations, Valideation, Simplicitly

Current Company: bennelijacobs&co.

Website: http://www.tendayiviki.com/

Why Innovation Fails 

A Startup is A Thesis • The term "thesis" comes from the Greek

 θέσις, meaning "something put forth", and refers to an intellectual proposition.

• "Dissertation" comes from the Latin dissertātiō, meaning "path".

A Path for Searching:

For a sustainable and profitable business model…

A Startup is Really Like a Doing Research Project:

We should not be making business plans, we should make research proposals.

The Startup Team is a Research Team:

All hands on deck, to learn what customers want…

And a sustainable/profitable way to deliver that value to them….

A Significant Contribution:

The significant contribution of a startup is building something people want…

Upon achieving product-market fit, a startup graduates…

A Product-Stage Centric Approach 

Ash Maurya (2012)Running Lean

http://www.runningleanhq.com

/

18

The Scientific MethodApplied to Innovation and Entrepreneurship

A Pseudoscience?

Associates with science and uses the similar language.

Often simplistic and relies on anecdotal evidence.

Uses research methods without properly thinking through their implications.

Lets Start at the BeginningWhere do startup ideas come from?

Getting to Plan B:

Analogs vs. Antilogs vs. Leaps of faith

• Personal Experience• Desk Research• Conversations with industry

experts• Market trend analysis• Porter’s five forces • PESTEL analysis

The scientific equivalent is the literature search and review.

A Product to Help People Choose Safe Places for Going Out

at Night 

26http://www.youtube.com/watch?v=QoAOzMTLP5s

Alexander Osterwalder (2012)http://www.businessmodelgeneration.com/

It is the leaps of faith to which we apply the scientific method to

test.

These can also be described as the riskiest

assumptions.

But Wait…Don’t just rush out of the building.

But Wait Still…Hypotheses must be falsifiable (Karl Popper, 1902-1994).

Falsifiable hypotheses have minimum success criteria.

Running Successful Experiments

Ash Mauryahttp://www.ashmaurya.com/2010/09/lean-startup-is-a-rigorous-process/

When You Have Clear Hypotheses: Think now about appropriate method to test.

On SamplingDon’t just rush out of the building and talk to anyone who will talk to you.

◦Random Sampling: Every individual in the population has an equal chance of being selected.oDrawing names out of a hat with all the names of students is an example of random sampling.oHowever, standing on the sidewalk and interviewing every fourth person is not random sampling. Because every person does not have an equal chance

of being selected.

◦Stratified Sampling: In this case criteria are set to ensure representation of particular groups. oFor example, if a population has 75% white people and 25% ethnic minorities. Within each category individuals are chosen randomly.

oThis techniques is used by opinion poll researchers. They usually consider age, race, gender, region and

other variables.

◦ Quota Sampling: This is when a researcher deliberately sets up sample

proportions that are different from the population. For example, if there is an interest in differences between

blacks and white on an opinion issue (50% blacks and 50% whites).

◦ Convenience Sampling: This is when a researcher uses the sample they can get

access to. For example, a sample of people at the mall or at the park that

day. Volunteer samples are also convenience samples.

◦ Purposive Sampling: This is when a researcher deliberately targets individuals

thought to be important for the research. Snowball sampling is an example of this. Also useful for focusing on early adopters.

Purposive SamplingAt the beginning we focus on early adopters.

39

40

not everyone is your customer

41

focus on Earlyvangelists

Has the problem.Is aware of having the problem.Has been actively looking for a

solution.Has put together a rough solution. Has or can acquire a budget.

42

A Conversation About Early Adopters for the

Safety at Night Product 

45

46

47

Customer Jobs to Be Done

http://www.youtube.com/watch?v=m3mVM_koAio

48

Sample Size:◦All things being equal, the larger the

sample the more reliable and representative it is.

◦ Sample sizes depend on both statistical and practical issues: Qualitative Research

Smaller samples (6+ people). Quantitative Research:

Larger samples (30+ people) Strength of effects:

Smaller samples work when you have large effects. Remember that early adopters are the approximation of

large effects.

Customer DevelopmentEarly conversations with customers to find out their problems tend to be exploratory.

Ash Maurya (2012)Running Lean

http://www.runningleanhq.com/

56

For early adopters set a high bar for problem

confirmation

57@robfitz

http://momtestbook.com//

Anybody will say your product is good if you bug them for long enough.

People also often don’t know the factors motivating their own behaviour.

If you asked your parents, could they lie to you?

/

@robfitzhttp://momtestbook.com//

The Mom Test

@paddayhttp://insideintercom.io//

60

Quiz:Good or Bad questions?

(by @robfitz)

61

Do you think it’s a good idea?

62

Do you think it’s a good idea?

63

Would you buy a product which solved this problem?

64

Would you buy a product which solved this problem?

65

How do you currently deal with this problem?

66

How do you currently deal with this problem?

67

Talk me through the last time you had this problem

68

Talk me through the last time you had this problem

69

How much would you

pay for this?

70

How much would you

pay for this?

71

How much money does this problem currently cost you?

72

How much money does this problem currently cost you?

73

Is there a budget for it?

74

Is there a budget for it?

75

Who else should I talk to?

76

Who else should I talk to?

77

You’re looking for emotions

Frustration. Excitement.

Things they care about

78

Ask about Frequency & Details

79

Ask about current solutions

80

Ask about past behavior

81

Quiz:Did we make progress?

You just walked out of a meeting with this result.

yes/no

(by @robfitz)

82

Sounds great. I love it!

83

Sounds great. I love it!

84

Brilliant, let me know

when it launches.

85

Brilliant, let me know

when it launches.

86

Compliment + Stalling tactic They don’t care

87

There are a couple of people I can intro you to, when you’re ready

88

There are a couple of people I can intro you to, when you’re ready

89

Partial commitment(Is there a valid reason they

can’t connect you now?)

90

I would definitely buy that!

91

I would definitely buy that!

92

DANGER!!!A promising signal, but

people can’t predict their own behaviour.

93

Compliments are NOT commitments

94

Use a script, but also allow interview to go off track

Ash Maurya (2012)Running Lean

http://www.runningleanhq.com/

96

Listen, don’t talk.

Encourage, but don’t influence.

Parrot back their answers.

Misrepresent to confirm

97

Always ask for referrals.

98

When do you stop customer development?

After you have completed your target sample size.

When there is no more incremental learning.

Experiment DesignAfter conversations there remains the need to have the customers perform some action

Ash Maurya (2012)RUNNING LEAN

VERIFY QUALITATIVELY

VALIDATE QUANTITATIVELY

@Robfitz

Learn

Confirm

What is an experiment? • The ability to deliberately manipulate one

variable (the independent variable), while holding other variables constant.

• Controlling confounding variables (e.g. through random assignment or matching participants).

• The use of control conditions in order to take baseline measures upon which to judge your results.

Is qualitative research scientific?

• The answer is: YES!

• Remember it is not tactics we are focused on, its principles.

• Each research method has its limitations.

• Qualitative methods may be affected by biased interpretations.

• Experimental methods may not allow deeper exploration of customer motivations.

A Tool for ResearchThe Minimum Viable Product

MVP = Experiment

/

The hypothesis/objective you are trying to learn.

from the target market segment you are trying to learn about.

and the minimum form that it takes to achieve that learning.

Patrick Vlaskovitz (2012)

http://vlaskovits.com/2012/09/apple-maps-debacle-and-minimum-viable-products/

The Minimum Viable Product

Mundane Realism: ◦How closely your research mirrors real life.

Experimental Realism:◦How closely your research place people in

the necessary mind-state to test your hypothesis.

Experimental realism is the goal of the MVP.

There is a Distinction Between:

MVP Examples

Doppelganger

Source: @ryanmaccarrigan

Conversation Starter

Source: Trevor Owens (@to2)

Concierge MVP

Source: @ryanmaccarrigan

Product Pitch MVP

Paper Prototype

Mechanical Turk

Source: @ryanmaccarrigan

Fake/Drop Features

Video MVP

http://www.youtube.com/watch?feature=player_embedded&v=7QmCUDHpNzE

AIRBnB Casestudy

http://www.udemy.com/lectures/growth-airbnb-case-study-joe-zadeh-76819

The Process 

Alexander Osterwalder (2012)http://www.businessmodelgeneration.com/

Talk about your business model and your findings from customer development.

Identify your assumptions about the solution customers want.

Describe one main solution.Transform that into a falsifiable solution

hypothesis.Develop two-three customer

development questions?

GET OUT OF THE BUILDING!!

Riskiest Assumption

Some Tools

Science is tentative…So what does this mean for innovation?

Don’t just A/B Test. REPLICATE AND EXTEND…

Other Interesting Issues

Ethics. Deception and Debriefing.

Determinism. What drives customers behaviour.

Measurement. Reliability and Validity. The Mom Test.

Study Design. Within Participants. Between Participants.

Statistical Analysis. Descriptive. Inferential.

THANK YOU