Iterating with a large community

2008

Photo credit: Flickr user ‘ mikejsolutions’

What’s the challenge?People have a sense of ownership, entitlement

Frequent users have established pathways

Expectations of rapid iteration from early phases

Niche interests dominate discussion

Impossible to satisfy everyone (incl your own team)

Difficult to contain scope

Workshop: “Iterating with a large community” 2008

Digg CommentsHuge and passionate participation

Several iterations already, lessons learned

Workshop: “Iterating with a large community” 2008

2005 2006 2007 2008

1. Analyze the challenge

2. Determine objectives

3. Define success

4. Early comps

5. User Test #1

6. Set expectations, get feedback

7. Full comps

8. User Test #2

9. Launch it!

10. Circle back and iterate again...

Workshop: “Iterating with a large community” 2008

What do they want?

Explicit feedback: email, comments, verbal

Implicit feedback: what are people doing?

Focus group test or usability test

Remember Henry Ford!

People often speak in specific solutions, but they’re

really identifying problems

Workshop: “Iterating with a large community” 2008

Tap into user feedback1

How would you find out ?

What are you solving?Break down the issues

Prioritize the problems

Avoid feature creep by staying focused on the

problems you defined

Workshop: “Iterating with a large community” 2008

Analyze the specific challenges2

Define successBe pragmatic. How will you know if you’re

successful?

Get everyone on the same page

Weather the storm after release

Workshop: “Iterating with a large community” 2008

Set measurable goals3

Workshop: “Iterating with a large community” 2008

Set measurable goals3

Goals for comments

Increase overall number of comments quantifiable

Increase number of commenters quantifiable

Improve page load performance yes/no

Fix scaling issues yes/no

Improve quality of discussions subjective

How can we measure success ?

Comp your changesEasier to discuss a draft than a raw idea

Do some revisions before sharing

Seek the best idea, not just the most obvious

Don’t present multiple comps for the sake of it

Workshop: “Iterating with a large community” 2008

Develop rough concepts4

How many people do this ?

Workshop: “Iterating with a large community” 2008

Develop rough concepts4

Go test your ideasGet early feedback on your ideas

Much easier to react at this stage

Gives ammunition in in-house debates

This is just the first usability test

Find low-hanging fruit issues

Workshop: “Iterating with a large community” 2008

User test round #15

Set expectationsMake a blog post with examples

Solicit more ideas from the community

Encourages sense of involvement

Genuinely good ideas arise

Do a video walkthrough

Announce a release date at your peril

Workshop: “Iterating with a large community” 2008

Set expectations6

Prepare full compsKeep your original goals in mind

Avoid last-second feature creep

Plan v2 and v3 features

Workshop: “Iterating with a large community” 2008

Refine based on testing and feedback7

Usability testActual task-based testing

This is your best chance to catch glaring issues

Leave time for this, it’s easy to be rushed

Workshop: “Iterating with a large community” 2008

User test round #28

Go live!Focus on bug fixing right away

Take initial feedback with caution

Don’t iterate immediately

Record your metrics

Success?

Workshop: “Iterating with a large community” 2008

Launch it9

Total comments increased by over 30%Unique commenters increased by around 20%Unique comments per person increased by ~15%Scaling fixed, but performance has weaknessesQuality of discussion appears to increase

Latest design saw increases again in these metricsVery positive explicit feedback was surprising

Results

Workshop: “Iterating with a large community” 2008

Launch it9

Prepare next iteration...

Workshop: “Iterating with a large community” 2008

Iterate again10

Questions

Workshop: “Iterating with a large community” 2008

?