Failing Fast & Learning Along the Way - Big Design 2013

192
FAILing FAST @jeremyjohnson & learning along the way

description

Mantras of startups: "fail fast", "move fast and break things", "keep shipping" - these are all great slogans, but unknown to many - these are really all about learning. It's about getting things in front of your customers early, and often. Watching - and learning. Finding what ideas were not quite as brilliant as you once thought - and finding this out as fast and cheap as possible. How are modern product teams making this happen? Where does User Experience and customer research fit in this model? Taking from Agile, Lean, and User Centered Design - this talk will go over the build-measure-learn process, and how you can start to shape your organization to move fast, without leaving your customers behind. This talk was given at Big Design 2013 #bigd13

Transcript of Failing Fast & Learning Along the Way - Big Design 2013

Page 1: Failing Fast & Learning Along the Way - Big Design 2013

FAILing FAST@jeremyjohnson

& learning along the way

Page 2: Failing Fast & Learning Along the Way - Big Design 2013

@jeremyjohnson

www.jeremyjohnsononline.com

#bigd13#leanux

Page 3: Failing Fast & Learning Along the Way - Big Design 2013
Page 4: Failing Fast & Learning Along the Way - Big Design 2013
Page 6: Failing Fast & Learning Along the Way - Big Design 2013

marketing / product / Development

Page 8: Failing Fast & Learning Along the Way - Big Design 2013
Page 9: Failing Fast & Learning Along the Way - Big Design 2013

UX 101un peu

Page 10: Failing Fast & Learning Along the Way - Big Design 2013

UX 101

we make things for

Page 11: Failing Fast & Learning Along the Way - Big Design 2013

UX 101

we make things for

http://500px.com/jeremyjohnson/sets/buenos_aires_2012

Page 12: Failing Fast & Learning Along the Way - Big Design 2013

UX 101

We work in ecosystems

http://500px.com/jeremyjohnson/sets/london_2012

Page 13: Failing Fast & Learning Along the Way - Big Design 2013

UX 101

we think visually

Page 14: Failing Fast & Learning Along the Way - Big Design 2013

UX 101

We learn through observation

http://500px.com/jeremyjohnson/sets/london_2012

Page 15: Failing Fast & Learning Along the Way - Big Design 2013

UX 101

we’re curious

http://500px.com/jeremyjohnson/sets/london_2012

Page 16: Failing Fast & Learning Along the Way - Big Design 2013

UX 101

Page 17: Failing Fast & Learning Along the Way - Big Design 2013

User-centered design1970s

http://en.wikipedia.org/wiki/User-centered_design

Page 18: Failing Fast & Learning Along the Way - Big Design 2013

User-centered design can be characterized as a multi-stage problem solving process that not only requires designers to analyze and foresee how users are likely to use a product, but also to test the validity of their assumptions with regard to user behavior in real world tests with actual users. Such testing is necessary as it is often very difficult for the designers of a product to understand intuitively what a first-time user of their design experiences, and what each user's learning curve may look like.http://en.wikipedia.org/wiki/User-centered_design

Page 19: Failing Fast & Learning Along the Way - Big Design 2013

Surveys

Usability testing

contextual inquiries

Interviews

Focus groups

Page 20: Failing Fast & Learning Along the Way - Big Design 2013

LEARNINGwith customers

Page 21: Failing Fast & Learning Along the Way - Big Design 2013
Page 22: Failing Fast & Learning Along the Way - Big Design 2013

“Agile methods like Scrum and XP both rely on a close and collaborative relationship and continual interaction with the customer – the people who are paying for the software and who are going to use the system.”

http://swreflections.blogspot.com/2012/02/agiles-customer-problem.html

2001

Page 23: Failing Fast & Learning Along the Way - Big Design 2013

LEARNINGwith customers

Page 24: Failing Fast & Learning Along the Way - Big Design 2013

How many people are able to learn by proxy (persona, market

research, etc...)?

Page 25: Failing Fast & Learning Along the Way - Big Design 2013

How many people are able to learn by watching your customer (analytics, pathing, heatmaps,

etc...) ?http://crave.cnet.co.uk/televisions/apple-tv-hint-pops-up-in-mountain-lion-trademark-filing-50008629/

Page 26: Failing Fast & Learning Along the Way - Big Design 2013

How many people are able to get in the same room with your (end)

customer?

http://tomboystyle.blogspot.com/2012/09/icon-pat-english.html

Page 27: Failing Fast & Learning Along the Way - Big Design 2013

http://blog.beatthebrochure.com/the-12-best-safaris-in-the-world/5597/lion-safari

How many people are able to visit their customers in their

natural habitat?

Page 28: Failing Fast & Learning Along the Way - Big Design 2013

LEARNINGwith customers

Page 29: Failing Fast & Learning Along the Way - Big Design 2013

LEARNINGwith customers

Page 30: Failing Fast & Learning Along the Way - Big Design 2013

what’s the least amount of time and money you can spend to learn something.

Page 31: Failing Fast & Learning Along the Way - Big Design 2013

“I dropped over $40k when I could have spent $100”

http://boondainc.wordpress.com/2013/04/18/i-dropped-over-40-grand-when-i-could-have-spent-100/

Page 32: Failing Fast & Learning Along the Way - Big Design 2013

Moving to lean...

Page 33: Failing Fast & Learning Along the Way - Big Design 2013

startup

Where the core component...

Page 34: Failing Fast & Learning Along the Way - Big Design 2013

...of Lean Startup methodology is the build-measure-learn feedback loop.

Page 35: Failing Fast & Learning Along the Way - Big Design 2013

build-measure-learn

Page 36: Failing Fast & Learning Along the Way - Big Design 2013

Spec-measure-learn

Page 37: Failing Fast & Learning Along the Way - Big Design 2013

How long does it take to build?

Page 38: Failing Fast & Learning Along the Way - Big Design 2013

hackDAY

Page 39: Failing Fast & Learning Along the Way - Big Design 2013

On May 22 this year 300 hackers converged in New York at TechCrunch Disrupt for a day and half long hack day before the conference itself started. At least one of the projects created at the hack day has now become an actual business, and has raised an angel round of funding from top tier investors.

http://techcrunch.com/2010/08/25/groupme-born-at-techcrunch-disrupt-secures-funding-and-launches/

Page 41: Failing Fast & Learning Along the Way - Big Design 2013
Page 42: Failing Fast & Learning Along the Way - Big Design 2013

LEARNINGwith customerswhile shipping

Page 43: Failing Fast & Learning Along the Way - Big Design 2013
Page 44: Failing Fast & Learning Along the Way - Big Design 2013
Page 45: Failing Fast & Learning Along the Way - Big Design 2013

mvp (keep it simple)

experiment via iterative prototyping

value customerfeedback

shipping / launching often

Page 46: Failing Fast & Learning Along the Way - Big Design 2013

UX

mvp (keep it simple)

experiment via iterative prototyping

value customerfeedback

shipping / launching often

we like simple

we build prototypes

User centered Design - that’s us! We’re want to launch!

Page 47: Failing Fast & Learning Along the Way - Big Design 2013

User-centered design

Agile

Lean

for designers

for Developers

for Product owners/teams

(Lean UX)

Page 48: Failing Fast & Learning Along the Way - Big Design 2013

But We’re not a startup...

Page 49: Failing Fast & Learning Along the Way - Big Design 2013
Page 50: Failing Fast & Learning Along the Way - Big Design 2013
Page 53: Failing Fast & Learning Along the Way - Big Design 2013
Page 54: Failing Fast & Learning Along the Way - Big Design 2013

core product team

product owner

ux designerdevelopers

Page 55: Failing Fast & Learning Along the Way - Big Design 2013

“mini-ceo”product owner

Page 56: Failing Fast & Learning Along the Way - Big Design 2013

“mini-ceo”product owner

shareholders

Page 57: Failing Fast & Learning Along the Way - Big Design 2013

increase conversion

reduce bounce

get more signups

increase usage

Page 58: Failing Fast & Learning Along the Way - Big Design 2013

Evaluate

Test

Prototype

Keep/Kill

(Build)

(Measure)

(Learn)

Page 59: Failing Fast & Learning Along the Way - Big Design 2013
Page 61: Failing Fast & Learning Along the Way - Big Design 2013

Large Organizations don’t have to be slow!

Page 62: Failing Fast & Learning Along the Way - Big Design 2013
Page 63: Failing Fast & Learning Along the Way - Big Design 2013

build-measure-learn

Page 64: Failing Fast & Learning Along the Way - Big Design 2013

Break two habits

Page 65: Failing Fast & Learning Along the Way - Big Design 2013

good at designing things

Page 66: Failing Fast & Learning Along the Way - Big Design 2013

good at designing things

good at determining what are the right things to build

=

Page 67: Failing Fast & Learning Along the Way - Big Design 2013

Make sure the customer can use

the feature

Make sure the feature has value for the customer

and design it accordingly

and prioritize it accordingly

sometimes... We should...

Page 68: Failing Fast & Learning Along the Way - Big Design 2013
Page 72: Failing Fast & Learning Along the Way - Big Design 2013
Page 74: Failing Fast & Learning Along the Way - Big Design 2013

Can we learn faster?

Page 75: Failing Fast & Learning Along the Way - Big Design 2013

Some amount of thinking around a project

Page 76: Failing Fast & Learning Along the Way - Big Design 2013

Build MVPLaunchLearn

Iterate(repeat)

Find customersperform researchsynthesize discoveries Build out prototypeTest with customer

a. B.

Page 77: Failing Fast & Learning Along the Way - Big Design 2013
Page 79: Failing Fast & Learning Along the Way - Big Design 2013

Systrom, Intuit founder Scott Cook, and Lean Startup author Eric Ries talked about the changes that have swept through product development in both big and small organizations. Many companies have moved from what's called "waterfall development" -- a method that relies on large engineering executing a carefully mapped-out plan -- to "lean" development, where creators move quickly to push out products and revise them on the fly.

"We thought about what we could do to iterate more quickly," Systrom said of Burbn's pivot. "People loved posting pictures on Burbn" -- so that's where they took the venture, jettisoning other planned features. Burbn now lives on only as an abandoned Twitter feed.

http://money.cnn.com/2011/09/13/technology/startups/instagram_burbn/

Page 80: Failing Fast & Learning Along the Way - Big Design 2013

LEARNINGwith customerswhile shipping

Page 84: Failing Fast & Learning Along the Way - Big Design 2013

Build MVPLaunchLearn

Iterate(repeat)

Find customersperform researchsynthesize discoveries Build out prototypeTest with customer

a. B.

Page 85: Failing Fast & Learning Along the Way - Big Design 2013

UX

mvp (keep it simple)

experiment via iterative prototyping

value customerfeedback

shipping / launching often

AGAIN

Page 86: Failing Fast & Learning Along the Way - Big Design 2013

Moving quickly

Page 87: Failing Fast & Learning Along the Way - Big Design 2013
Page 90: Failing Fast & Learning Along the Way - Big Design 2013

http://littlebigdetails.com/

Page 93: Failing Fast & Learning Along the Way - Big Design 2013

BALANCE

Page 94: Failing Fast & Learning Along the Way - Big Design 2013

embracing and Agile

Page 95: Failing Fast & Learning Along the Way - Big Design 2013

Through discovery

Find what has value

Quickly test

Quickly launch

Via Prototyping

with customers

with development

With product owner

Page 96: Failing Fast & Learning Along the Way - Big Design 2013

LEARNINGwith customers

Page 97: Failing Fast & Learning Along the Way - Big Design 2013

“Rather than focus on artifacts, we focus on prototypes and validating those prototypes in Discovery, with the added benefit that the prototype serves as the spec for Delivery.”

http://www.svproduct.com/dual-track-scrum/

Page 98: Failing Fast & Learning Along the Way - Big Design 2013

Documentation = bad

rough, quick, iterative, prototype = good

Page 99: Failing Fast & Learning Along the Way - Big Design 2013

Getting closer, quicker to the actual experience

Page 102: Failing Fast & Learning Along the Way - Big Design 2013

1. Drive: UX practitioners are part of the customer or product owner team

2. Research, model, and design up front - but only just enough

3. Chunk your design work

4. Use parallel track development to work ahead, and follow behind

5. Buy design time with complex engineering stories

6.Cultivate a user validation group for use for continuous user validation

7. Schedule continuous user research in a separate track from development

8. Leverage user time for multiple activities

9.Use RITE to iterate UI before development

10.Prototype in low fidelity

11.Treat prototype as specification

12.Become a design facilitator

http://agileproductdesign.com/blog/emerging_best_agile_ux_practice.html

- 2008

(not new)

Page 103: Failing Fast & Learning Along the Way - Big Design 2013

agilea quick word about

Page 105: Failing Fast & Learning Along the Way - Big Design 2013
Page 106: Failing Fast & Learning Along the Way - Big Design 2013

agile lean UX/

everyone is involved!

Page 107: Failing Fast & Learning Along the Way - Big Design 2013

agile, better than waterfall

Page 108: Failing Fast & Learning Along the Way - Big Design 2013

agile, shows value faster

Page 109: Failing Fast & Learning Along the Way - Big Design 2013

agile, can work well with UX

Page 110: Failing Fast & Learning Along the Way - Big Design 2013

agile, makes better software

Page 111: Failing Fast & Learning Along the Way - Big Design 2013
Page 112: Failing Fast & Learning Along the Way - Big Design 2013
Page 113: Failing Fast & Learning Along the Way - Big Design 2013

valuableUsable

Enjoyable

Page 114: Failing Fast & Learning Along the Way - Big Design 2013

valuableUsable

Enjoyable

are we building the right things?

Is it easy to use?

Do they want to use it?

Page 115: Failing Fast & Learning Along the Way - Big Design 2013

valuable Usable

Enjoyable

Page 116: Failing Fast & Learning Along the Way - Big Design 2013

determine value,create experience,

repeat

Page 117: Failing Fast & Learning Along the Way - Big Design 2013

how

Page 118: Failing Fast & Learning Along the Way - Big Design 2013

“We gathered our designers, our product folks and our engineers and took over a few conference rooms and began to operate like a startup. Design was done on whiteboards and coded in real time. Usability tests were weekly so the pace was fast and furious. But we were able to try dozens of experiences across desktop, tablet and mobile in the time that would have taken years at PayPal before. Build/Test/Learn became our mantra.”

http://looksgoodworkswell.blogspot.com/2012/09/why-you-should-work-with-me-at-paypal.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+LooksGoodWorksWell+%28Looks+Good+Works+Well%29

-

Page 119: Failing Fast & Learning Along the Way - Big Design 2013

Build/Test/Learn

Page 121: Failing Fast & Learning Along the Way - Big Design 2013
Page 122: Failing Fast & Learning Along the Way - Big Design 2013

work differentwork different

Page 123: Failing Fast & Learning Along the Way - Big Design 2013

COMMONPROBLEMS

Page 124: Failing Fast & Learning Along the Way - Big Design 2013

Can’t get started?- problem -

Page 125: Failing Fast & Learning Along the Way - Big Design 2013

Ship often. Ship lousy stuff, but ship. Ship constantly.

http://99u.com/tips/6249/Seth-Godin-The-Truth-About-Shipping

Page 126: Failing Fast & Learning Along the Way - Big Design 2013
Page 127: Failing Fast & Learning Along the Way - Big Design 2013

Spend too much time planning?

DOn’t talk with your customers?

Find it hard to strip out what’s not valuable?

- problem -

Page 129: Failing Fast & Learning Along the Way - Big Design 2013

“The timing of long- range plans is screwed up too.You have the most information when you’re doing something, not before you’ve done it. Yet when do youwrite a plan? Usually it’s before you’ve even begun.That’s the worst time to make a big decision.”

http://37signals.com/rework

Page 130: Failing Fast & Learning Along the Way - Big Design 2013

Large team

- problem -

Page 131: Failing Fast & Learning Along the Way - Big Design 2013

Keep your team small. Smaller than that. No team at all if you can help it.

http://99u.com/tips/6249/Seth-Godin-The-Truth-About-Shipping

Page 132: Failing Fast & Learning Along the Way - Big Design 2013

A throwback to their days with Jeff Bezos at Amazon, projects are assigned to "two pizza teams," groups of engineers small enough for them to be fed on two large pies. "We want the team to be flat and allow everyone to communicate with each other," Rajaraman says.

http://www.fastcompany.com/1811934/walmartlabs-brings-two-pizza-team-startup-culture-walmart-empire

Page 133: Failing Fast & Learning Along the Way - Big Design 2013
Page 135: Failing Fast & Learning Along the Way - Big Design 2013

Only launch x times a year?Looking for perfection?

Try to jam too much into a product?

- problem -

Page 137: Failing Fast & Learning Along the Way - Big Design 2013

https://frontdeskhq.com

Page 138: Failing Fast & Learning Along the Way - Big Design 2013

“Great companies focus on their users and ship great products.”http://www.aaronklein.com/2012/02/why-facebook-is-worth-100-billion/

Page 140: Failing Fast & Learning Along the Way - Big Design 2013

“real artist ship”- steve jobs

http://gloriamarie.com/stay-focused-and-keep-shipping

Page 141: Failing Fast & Learning Along the Way - Big Design 2013

It’s going to cost too much to try that out.

- problem -

How do we know our customers will want this?

Page 142: Failing Fast & Learning Along the Way - Big Design 2013

The Wizard Of Oz Techniques For Social Prototyping – You don’t need to build everything at first. You can be the man behind the curtain. Krieger says him and Systrom tested an early version of a feature which would notify you when friends joined the service. Instead of building it out, they manually sent people notifications “like a human bot” saying ‘your friend has joined.’ It turned out not to be useful. “We wrote zero lines of Python, so we had zero lines to throw away.”

http://techcrunch.com/2012/11/30/instagram-co-founder-mike-kriegers-8-principles-for-building-products-people-want/

- Mike Krieger, Instagram’s founder

Page 144: Failing Fast & Learning Along the Way - Big Design 2013

404 testing

NEW FEATURE X

Page 146: Failing Fast & Learning Along the Way - Big Design 2013
Page 148: Failing Fast & Learning Along the Way - Big Design 2013

It was an MVP (Minimal Viable Product). I skipped a bunch of features I figured I would implement later. First I wanted to see if people would use it and how they would use it.

Implementing user accounts (in Rails) would take me 2 weekends of work; registration, accounts, saving lists, removing lists, tracking, designing screens, edge cases etc.

I didn’t want to spend the time if it turned out no one signed up so I ran an experiment.

I dropped in a link on the top of the page that said “Sign up to save multiple lists.” and tracked the number of clicks it got with Mixpanel.

(...)

http://www.leemunroe.com/lean-product-development-validate-feature-ideas/

Page 150: Failing Fast & Learning Along the Way - Big Design 2013

What is the cheapest, fastest way to learn?

Page 151: Failing Fast & Learning Along the Way - Big Design 2013
Page 152: Failing Fast & Learning Along the Way - Big Design 2013
Page 153: Failing Fast & Learning Along the Way - Big Design 2013
Page 154: Failing Fast & Learning Along the Way - Big Design 2013

Just some of the issues around software development

Page 155: Failing Fast & Learning Along the Way - Big Design 2013

Racing to the right ideas

Page 156: Failing Fast & Learning Along the Way - Big Design 2013

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

Idea or discovery backlog

Page 157: Failing Fast & Learning Along the Way - Big Design 2013

The Discovery track is all about quickly generating validated product backlog items, and the Delivery track is all about generating releasable software.

http://www.svproduct.com/dual-track-scrum/

- marty cagan

Page 158: Failing Fast & Learning Along the Way - Big Design 2013

http://www.sebastiangreger.net/writings/concept-design-in-agile-environment/

Iteration Iteration Iteration Iteration

discovery discovery discovery discovery

Implement Implement Implement Implement

design

build

discovery backlog

Page 159: Failing Fast & Learning Along the Way - Big Design 2013

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

Page 161: Failing Fast & Learning Along the Way - Big Design 2013

prototype

Page 162: Failing Fast & Learning Along the Way - Big Design 2013

fast rough keep moving

Page 163: Failing Fast & Learning Along the Way - Big Design 2013

test rough prototypes(usually built within a week or less)

Page 165: Failing Fast & Learning Along the Way - Big Design 2013

lab setting - 6 participants

9:30am

10:30am

11:30am

1:30pm

2:30pm

3:30pm

Page 166: Failing Fast & Learning Along the Way - Big Design 2013
Page 167: Failing Fast & Learning Along the Way - Big Design 2013
Page 168: Failing Fast & Learning Along the Way - Big Design 2013

valuableUsable

Enjoyable

determine

Page 169: Failing Fast & Learning Along the Way - Big Design 2013

clicktest / survey / Etc...

Page 170: Failing Fast & Learning Along the Way - Big Design 2013

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

ideas

Page 171: Failing Fast & Learning Along the Way - Big Design 2013
Page 172: Failing Fast & Learning Along the Way - Big Design 2013
Page 173: Failing Fast & Learning Along the Way - Big Design 2013

x

x

Page 174: Failing Fast & Learning Along the Way - Big Design 2013

Core team makes decisions

Done!

the core team

Page 175: Failing Fast & Learning Along the Way - Big Design 2013

...it is collaborative – the product manager, designer and lead engineer are working together, side-by-side, to create and validate backlog items.

http://www.svproduct.com/dual-track-scrum/

- marty cagan

Page 176: Failing Fast & Learning Along the Way - Big Design 2013

ideas

failed usabilityfailed to understandfailed to find value

ideas

ideas

Page 178: Failing Fast & Learning Along the Way - Big Design 2013

ideas

refine retest

Page 179: Failing Fast & Learning Along the Way - Big Design 2013

ideas

ideas

Ready for development

Page 180: Failing Fast & Learning Along the Way - Big Design 2013

ideas

ideas

What’s your kill rate? Ship that bad boy!

Page 181: Failing Fast & Learning Along the Way - Big Design 2013

Did I mention this happens within a week?

(or less)

Page 182: Failing Fast & Learning Along the Way - Big Design 2013

Get moving!

Page 183: Failing Fast & Learning Along the Way - Big Design 2013

LEARNLEAN

Page 184: Failing Fast & Learning Along the Way - Big Design 2013
Page 185: Failing Fast & Learning Along the Way - Big Design 2013

combined product teams

#1

Page 186: Failing Fast & Learning Along the Way - Big Design 2013

one ux designer per team

#2

Page 187: Failing Fast & Learning Along the Way - Big Design 2013

rough, fast, iterative prototyping

#3

Page 188: Failing Fast & Learning Along the Way - Big Design 2013

Getting in front of customers weekly

#4

Page 189: Failing Fast & Learning Along the Way - Big Design 2013

build/test/learn

#5

Page 190: Failing Fast & Learning Along the Way - Big Design 2013

lean, agile, prototyping, shipping, ux design master.

in no time at all, you too can be a...

Page 192: Failing Fast & Learning Along the Way - Big Design 2013

thanks!@jeremyjohnson

www.jeremyjohnsononline.com

Slides here