Getting Started

Conducting User Research

Carol Smith

@carologic

Agile 2010

August 12, 2010

What You’ll Learn

• Quick and cheap methods, integrated into Agile.

• Understand needs and behaviors.

• Effectively share and communicate information.

2

DESIGN FOR EVERYONE

IS

IMPOSSIBLE

3

Who are you

developing for?

What do they

need to do?

4

Scope for Success

• You’ve heard it takes time, energy, and budget

and ____________ to:

- Research and discover

- Document

- Analyze

- Understand

5

And we have

constraints…

6

Focus Your Efforts Until…

7

Get to 80%

We Are 80% Sure We Know…

• primary user tasks.

• user’s goals.

• Prioritize as needed with:

- why we are doing this (vision).

- business needs

- etc.

• Have awareness of what we don’t know (yet).

8

Where do I start?

9

Who are your users?

10

Same Job Title, May Differ in…

UserExperience

Frequency of use

Priority of tasks Characteristics

Motivations and attitude

Expectations

11

12

Begin the Research

• Interview the experts:

- Customer Service

- Marketing (Web statistics)

- Training

- Sales/Business development

13

Begin the Research (cont)

• Who are the users?

• How many are there?

• Common complaints?

• Show stoppers?

• Understand:

- Assumptions and stereotypes

- Differences between users

14

Find Out About…

• Tasks: frequency, importance, complexity

• Environment: location, physical ability, situational

limitations

• Experience: job title and subject matter, product use

• Technology: Type of computer, mobile device, Internet

connection

15

Define Primary & Secondary Users

• Separate by:

- Needs

- Goals (Why will they use the product?)

- Environment (Where will they use it?)

- Context (When will they use it?)

- How else do they differ?

Now You Have User

Groups

[perhaps very loosely

defined]

16

Model Project

• Card Sort

• What we know

• Profile/Personas

• Tasks / Mental Models

• User Stories

• Wireframes

Discovery & Goal Setting

• Sprint(s)

• What we don’t know - Interview or Observations

• Usability Test Prototype

1st Prototype• Sprint(s)

• More research as questions arise

• Usability Test Prototype

2nd

Prototype

17

Share What You Learn

18

19

Personas

• Help guide decisions about:

- Navigation

- Features

- Design

• Archetype, based on research.

20

Task Analysis

21

Example of a Task Analysis by Todd Zaki Warfel from his Agile2010 presentation

"Opening the Kimono a look behind the design process."

Mental Models…

• Help the team

- understand user’s reasons for doing things.

- design for user’s needs and behaviors.

• Help management

- Prioritize content and solutions.

- Create solutions for opportunities.

Mental Model

Behaviors, thoughts, goals or intents of users

Content and/or solutions provided

Mental Space (goal or intent)

Opportunity

Actionable Gap AnalysisMental States & High Level Tasks

•Make appointment

•Plan for future appointments

•Coordinate appointments

Suggested Solutions

•FAQ on how to make an appointment

•Online Request for Appointment

online

•800 number prominent

Change Situation

Are We Confident?

25

26

27

Confirm Assumptions

• Representative users who DO the tasks.

• Visionaries, leaders, perhaps.

Observations

28

Go to the user

29

30

Why Observe?

• Great way to understand your user’s situation

- Find “cheat sheets” and other artifacts.

- Learn real process they use.

- Number and type of interruptions.

- Find out more about them as people.

31

Sit Back and Watch

• Arrive when they will be doing related tasks.

• Observe for as long as needed:

- 1/2 hour each - quick repetitive tasks

- >1 hour for longer processes

• Stay out of their “space” and don’t interrupt.

• Take photos and videos.

32

Take Detailed Notes

• Write down questions and when they occurred.

• Look for patterns and differences:

- Style of tasks

- Order of operations

- Environment

33

Clarify Observations

• After observation ask about:

- Why they do task?

- What is their goal?

- How typical was this process?

- Parts of the process you found confusing.

Collect, Copy, and/or

Photograph Artifacts!

34

Interviews

35

36

Interviews

• Discover facts and opinions

- Structured

- Open-ended

- Combination

• Scripts

- Promote consistency

- Memory tool for facilitator

37

Facilitate

• Unbiased questions

• Remain passive

• Ask your question and let them talk

• Encourage participant to elaborate

• Listen for vocalizations

• Watch non-verbal gestures

Silence is Golden

38

Wait for it…

Wait for it…

Wait for it…

Activity: Interview

• Pick a partner and interview about common subject:

- Washing clothing (size of family, separation, etc.)

- Getting gas (type of vehicle, when get it, etc.)

- Make travel arrangements (what’s important to them)

- Getting ready in morning (order of activities)

39

Interviews

• How many of you started to hear a pattern?

• How many more interviews would you need to verify?

40

41

Focus Groups

• Group interviews

• Reveal perceived desires, experiences and priorities

- What and how they think

• Good moderator is key

- Group can be dominated, biased, coerced

- May hear from minority of participants

• Relatively inexpensive

Usability Testing

(Demo, Acceptance Test)

42

43

Usability Testing

• Target users perform representative tasks.

• Understand impact on users.

• Measure success.

• Types of usability tests:

• Formal - usability measures

• Informal – qualitative data

• Lab, remote, on-site (home, conference, etc.)

• Observed by stakeholders!

Test Anything

at Anytime!

44

45

Card Sorting

Gaffney, Gerry. (2000) What is Card Sorting? Usability Techniques Series,

Information & Design.

http://www.infodesign.com.au/usabilityresources/design/cardsorting.asp

Optimal Sort, Optimal Workshop - http://www.optimalworkshop.com/

Demo: https://livedemo.optimalworkshop.com/optimalsort/supermarketdemo

How Do I Find

Participants?

46

47

Create a Screener

• Guide that helps determine who will participate.

• Ask people to describe, then get details:

- Highest level of education.

- Computer activities.

- Web use.

• People who pass the screener should closely match

your user group definition

48

Hire a Recruiter

• Allows you to focus on activity.

• Can tell if person will be a good participant.

• May already have a list they can start with.

• Good recruiters:

- find right participants.

- give regular updates.

- take care of directions, confirmations, incentives, etc.

49

If You Must Do it Yourself...

• Go where users go and intercept

• Online user groups

• Professional organizations

• Craigslist

• Use tools like Ethnio through your website

http://ethnio.com

• Always conduct final recruiting by phone.

• Ask questions that force them to talk.

• Don’t recruit non-talkers.

50

Number of Users to Test

• As many as possible (rarely statistically significant)

• Usability Testing Research (in 1990’s)

- 5 from distinct sub-group of the user population will

yield 80% of the findings (Nielsen, Virzi, Lewis)

- Assumes expert has reviewed for obvious issues

• Recommend:

- Early tests with 8 – 12 users per user group

- Iterative testing (3 per day, iterate, 3 new users)

Barnum, Carol M. (Jan. 2003). What’s in a Number? STC Usability SIG Newsletter, Usability Interface. http://www.stcsig.org/usability/newsletter/0301-number.html Retrieved: 20080323

51

52

Welcome & Prepare

• Participation will help team and is appreciated.

• Purpose of research.

• Expectations of the participant.

• Sign paperwork:

- Non-Disclosure Agreement(s)

- Consent Form

Your Goal

53

54

We’re Looking for Patterns

• Identify repetition

• After pattern is found, continuation of study

- Adds cost

- Delays reporting

- Low probability of many new findings

Update Communications

55

Model Project

• Sprint(s)

• Continue Research

• Usability Test Prototype

3rd

Prototype

• Sprint(s)

• Prepare for next stage/project

• Usability Test Prototype

4th

Prototype • Sprint(s)

• Prepare for next stage/project

• Usability Test Prototype

5th

Prototype

56

Do UX Early & Often

57

58

In Closing

• Get to your users

• Find out:

- Goals

- Tasks

• Share the information with your team

59

Recommended Readings

• www.UsabilityBoK.org

60

References• Cato, John. User-Centered Web Design. Addison Wesley Longman; 2001.

• Gaffney, Gerry. (2000) What is Card Sorting? Usability Techniques Series,

Information & Design.

http://www.infodesign.com.au/usabilityresources/design/cardsorting.asp

• Hackos, JoAnn T., PhD and Redish, Janice C. User and Task Analysis for

Interface Design. Wiley; 1998.

• Henry, S.L. and Martinson, M. Evaluating for Accessibility, Usability Testing in

Diverse Situations. Tutorial, 2003 UPA Conference. (Activity)

• Kuniavsky, Mike. Observing the User Experience: a Practitioner's Guide to User

Research. Morgan Kaufmann, 2003.

• Mandel, Theo. The Elements of User Interface Design. Wiley; 1997.

• Nielsen, Jakob and Robert L. Mack. Usability Inspection Methods. John Wiley

& Sons, Inc. 1994.

• Powell, Thomas A. The Complete Reference: Web Design. Osborne/McGraw-

Hill; 2000.

• Rubin, Jeffrey. Handbook of Usability Testing. John Wiley & Sons, Inc.; 1994.

61

Thank You!

• Carol Smith

• Midwest Research, LLC

• http://www.mw-research.com

• Twitter: @carologic

• Blog: http://www.AskaUser.com

• Email: carol@mw-research.com