Qualitative  User    Research  MethodsMattias  Arvola

2

The process has to begin with some kind of immersion in context and exposure to people who are perhaps experiencing that aspect of life right now. It starts by finding out where we are gonna go, who we are going to speak to, what is their world and really stepping into their shoes. For us to start a design project without that is a bit like cutting of our arms and legs and probably our head as well, we don't know where to start in that case. – Anonymous User Experience Designer

3

4

• Visit your users. • Speak with them. • Write down what you have seen and heard.

5

Time

Cost  for    Change  Requests

6

Time

Impact  of  UX

Cost  for    Change  Requests

7

• Value is created in use by your users. • Knowing them, their goals and activities is vital for

success.

Aims  of  User  Research

8

• Gain insights on: • What we should to design/build/develop • Why? • How to do it?

Research  Questions

9

• Qualitative • What? • How? • Why?

• Quantitative • How much? • How long? • To what degree? • What affect does X have on Y?

Qualitative  Research

10

Interpretative  research

11

• Research is interpretative when the understanding of human sense making is gained through the study of social constructions (e.g. language, shared meanings, consciousness, artefacts etc.) as a situation emerges.

The  Hermeneutical  Circle

12

Parts

Whole

Characteristics  of  Qualitative  Research

13

• No sharp boundaries of study objects • Careful contextualization of the instance • Transferability rather than generalizability • Cooperation between researcher and participants • Purposeful sampling • Alternative interpretations • Suspicion of interpretations • Going beyond what is said and done to what is

intended and motivated

Validation  Strategies14

• Prolonged engagement and observation • Building trust • Learning the culture • Checking misinformation

and distortions • Finding the focus

• Triangulation • Researcher • Theory • Method

• Peer review, debriefing and external audits

• Clarifying what subjective things that have likely shaped the the interpretation • past experiences, • biases • prejudices • orientations

• Member checking • Rich and thick description • Revise working hypothesis

Negative cases with disconfirming evidence

Preparations

15

Initial  Framing

16

• What is the project about? • Finding the target group • Not the most representative, but the most

important users • Who’s needs are the most important for us to

succeed? • Who might have something interesting to say? • Who are the ones most affected by the project? • Prioritise

Sensitizing  Persona  hypothesis

17

• Imagine mainstream and extreme users • What do they do? • In what situations? • How do they do it? • Why do they do it? • What characterizes them in terms of motivations,

knowledge and skills. • What do they lova an hate?

How  many  should  we  meet?

18

• Breadth and depth vs. cost • Three participants from three places/contexts • As different as possible but within the market

segment • Recruiting takes time

Ethics  and  Informed  Consent

19

• Knowing that you are observed • Understanding what it means • Potential risks • Gains • Anonymity or not

• Permission to record • Provide the participants with the results • Introduction in writing

Gathering  Data

20

Setting  a  Focus

21

• Getting information and inspiration for design that gives leverage

• Start with the brief and discuss it with clients • Find things to handle in the project

The  Persona  and  Scenario  Sheet

22

• A checklist with themes to cover • The scenario part: What people do • The persona part: How people are • Who does what, where, when, how, and why (5WH) • Download from:

http://www.ida.liu.se/~matar/tools.en.shtml

Goals and driving forces

Quote

Portrait

Name:

Description:

Age:

Role:

Sex:

Income:

Hometown:

Likes

Background

Skills and knowledge

Dislikes

Habits

Family and contacts

Experiences

Persona and scenario sheet. Mattias Arvola 2014.

Foundation for personas

Project:

Date:

Researcher:

An ordinary day starts with…

Persona and scenario sheet. Mattias Arvola 2014.

An ordinary day for the persona

Project:

Date:

Researcher:

WHO does it and who are involved?

WHAT is done?

WHEN and WHERE is it done?

HOW is it done?

WHY is it done? (participants' motivation and researchers' interpretation)

Persona and scenario sheet. Mattias Arvola 2014.

Foundation for scenarios

INTERPRETATIONSOBSERVATIONS AND FACTS

Weaknesses to address

Strengths to emphasize

Constraints to deal with

Project:

Date:

Researcher:

What  to  look  for  • Observations and facts • Opportunities for change • Things to preserve and highlight • Constraints • What characterizes a good/bad situation? • Use a protocol for semi-structured interviews and

observations (the persona and scenario sheet?) • Phenomena and patterns

26

Field  notes

27

• Separate observations and interpretations • Use pseudonyms • Quotes • Sketching with comments and detail studies • Note the background, steps and contexts of activities • Write the notes up as soon as possible

Recordings

28

• Video, audio or neither • Quotes • Intrusiveness • Transcription and levels of it • Clips of episodes • No replacement for field notes

Interviews

29

• Open questions that make participants tell stories and explain • Tell me about the first time you… • Tell me about a critical situation when matters

were brought to a head… • Could you describe when you could not… • Tell me about the last time you… • Tell me about when it went really well or really

bad…

Keep  them  talking30

• How do you mean when you say…

• What are you doing now?

• How do you think about this situation?

• What are you taking notice of in this situation?

• Are there any rules of thumb for how to proceed now?

• What information du you use here, how do you get it, and what do you do with it?

• Is there anything that makes this particularly difficult?

• Why is that important? • Have I understood you

correctly if I were to say…

WHY?

31

• Difficult question. • Users may start to speculate, but sometimes they

know exactly why. • Treat the answer with a certain degree of suspicion

and triangulate with observational data.

What  to  avoid

32

• Questions that participants cannot answer to themselves

• Closed questions • Do you like this web site? • Do you think this web shop is slow?

• Leading questions • Doesn’t this button look like a search button?

(leading) • What does this button do? (less leading) • What do you think about now? (inquiring)

Avoid  asking  users  to  speculate  about  the  future

33

• Talk about past events and what they did do. They will most likely keep along the same lines

• Only ask them to speculate on alternatives in relation to a specific event

Observations

34

• Level of participation • Fly on the wall • Shadowing • Secret agent • Apprentice

Observation  focus

35

• What order do people do things? • What cognitive activites (problem solving, memory,

perception, decision making etc.) are involved? • What info is used and how is it displayed? • What tools are used and what do they contribute

with? • Are there any workarounds people have developed to

compensate for constraints in tools and workplace? • What kinds of errors can be made? • What other kinds of support could be useful?

Contextual  Inquiry

36

• Combining interviews and observation • Being an apprentice • In context, in place • Show and tell • Figure out what is important for a participant • Focus points, but no strict protocol: semi-structured

Analysis

Affinitity  diagramming

38

• Categorisation • Finding themes • Sorting cards or sticky notes • Alone, in the design team, or with stakeholders

39

1. Read all notes and highlight important passages

2. Every participant reads the highlighted episodes aloud and writes them down on sticky notes. Code the origin

3. Get them up on the wall 4. Group them by identifying

similarities or relations. In silence. Keep moving notes until nobody wants to move any more

5. Discuss what binds the groups together: the shared meaning

6. Name the groups using specific notes creating categories. Categories that belong together form themes. There might be sub-categories too.

7. Draw lines between themes, categories and sub-categories to create a hierarchy.

Idea

Idea

Idea

Idea

Idea

Idea

Idea

Idea

Idea

Idea

Idea

Idea

Idea Idea

Category  1

Category  2

Category  3

Category  4

Theme  2Theme  1

Reviewing  the  Affinity  Diagram

41

• What is surprising here? • What did we expect to find that we didn’t find? • What do data tell us that we didn’t know before? • What in the data make us look at things differently? • Why is a certain pattern, or a certain anomaly,

appearing?

Modelling  the  User  Research

42

• Think: Concept mapping • Feel: UX Analysis with IPA • Do: Task Analysis • Personas, Scenarios, Storyboards and Customer

Journey Maps • Setting design objectives

Concept  Mapping

43

hZp://www.dubberly.com/wp-­‐content/uploads/2008/06/ddo_brand.pdf  

hZp://cmap.ihmc.us/publica`ons/researchpapers/theorycmaps/theoryunderlyingconceptmaps.htm

Method:  Concept  Mapping

46

1. Focus question. How is thunder formed? 2. Identify the 15–25 most important concepts in the

domain and place them in ”the parking lot” 3. Write them down on sticky notes and move them

around to form a first map 4. Search for cross-links between concept in different

parts of the map 5. Revise the map, make sure concept-link-concept

create propositions 6. Work on the typography and the layout

UX  Analysis

47

How  do  the  users  feel  and  experience  the  various  aspects  of  the  situation?

48

Ethics!

Build quality!

Organization!

Functionality!

Communicaton!

Impact!

Design proposal:__________________________________ !! ! !! ! !!Reviewer:__________________________________ !!IxDQI—Interaction Design Quality Indicator. Mattias Arvola. 2011-04-15!

-1!

0!

1!form and material!

character and innovation!

experience of activity!

symbolics!

cooperation and coordination!

usefullness!

usability!

operations!

business!

possibilities!

constraints!

habits!

power!

norms!

Aesthe9c

Technical

Method:  Interpretative  Phenomenological  Analysis

50

• Decide what users (3–5) to make a more in-depth analysis of

• Read and re-read. Carefully.

• Make first notes: • What things are important

to the user (relations, processes, places, events, values and principles)?

• What do they mean for the user and how are they experienced?

• Why are they important and why are they experienced in this way (how do they talk about it and what is the context)?

• What underlying abstract concepts can be used to understand the meaning of their statements?

Develop  themes

51

• Form Statements about: • how things are experienced • what those experiences mean for the participant

• Order them according to their affinity • What is similar across participants? • What is different? Why?

Task  Analysis

52

A  task  analysis  describes

53

• Who does what, when, where, how and why • How often? • How long? • How critical? • What works well? • What doesn’t work well?

Construct  a  task  set  in  a  flow  chart

54

Peel  the  task  set  into  layers

55

Remove  redundant  dependencies

56

Personas

57

• Precise descriptions of hypothetical individuals based on facts about potential users.

• Portrayed like a real living person with a name and a face to increase empathy

• Provides a shared understanding on whom to design for

• A tool for communication in the team and with clients • What would Bob the bank clark do? • Shaped around goals • Primary and secondary personas

Matthew Johnson Program Staff Director, USDA

51  years  old  

Married  

Father  of  three  children  

Grandfather  of  one  child  

Has  a  Ph.D.  in  Agricultural  Economics.  

Goals  and  tasks  

He is focused, goal-oriented within a strong leadership role. One of his concerns is maintaining quality across all output of programs. Spends  his  work  time  

• Requesting and reviewing research reports,

• preparing memos and briefs for agency heads, and

• supervising staff efforts in food safety and inspection.

Environment  

He is comfortable using a computer and refers to himself as an intermediate Internet user. He is connected via a T1 connection at work and dial-up at home. He uses email extensively and uses the web about 1.5 hours during his work day. Quote  

“Can you get me that staff analysis by Tuesday?”

http://www.usability.gov/how-­‐to-­‐and-­‐tools/methods/personas.html  

Method:  Personas

60

1. Identify roles to create personas for 2. Go through the data from users in the selected role 3. Identify variables on which the users differ and set up

bipolar scales 4. Place the users on the scales 5. Identify patterns by looking for users that go together

on several variables. They will form personas 6. Define goals for the interviewed users whom were

close to each other and put them down in the persona

61

7. Clarify distinctions between personas and add details 8. Repeat the procedure with the other personas if

needed 9. Group and prioritize the personas 10.Develop storytelling and other communication by

giving the persona a face and name. Make it come alive.

Placing  on  scales  and  finding  patterns

62

ANVÄNDER ENBART EGEN

DATOR

ANVÄNDER ENBART OFFENTLIG DATOR

ANVÄNDER OFTA MASKIN-

ÖVERSÄTTNING

ANVÄNDER ALDRIG MASKIN-ÖVERSÄTTNING

LÄSER SNABBT

LÄSER LÅNGSAMT OCH METODISKT

2 4 1 3 5

3 5 1 2 4

5 3 1 4 2

Uses  only    own  computer

Uses  always    machine    

translation

Reads    quickly

Uses  only    public  computers

Uses  never    machinetranslation

Reads  slow    and  carefully

Triading  the  users

63

• Go trough data from three randomly selected users at the time

• Find one characteristic that sets one apart from the other two and write down as a bipolar scale

• Take a new randomly selected triad and do the same • Go through ten triads • Discuss and come to consensus on which variables to

include

Scenarios64

Mr. and Mrs. Macomb are retired schoolteachers who are now in their 70s. Their Social Security checks are an important part of their income. They've just sold their big house and moved to a small apartment. They know that one of the many chores they need to do now is tell the Social Security Administration that they have moved. They don't know where the nearest Social Security office is and it's getting harder for them to do a lot of walking or driving. If it is easy and safe enough, they would like to use the computer to notify the Social Security Administration of their move. However, they are

somewhat nervous about doing a task like this by computer. They never used computers in their jobs. However, their son, Steve, gave them a computer last year, set it up for them, and showed them how to use email and go to websites. They have never been to the Social Security Administration's website, so they don't know how it is organized. Also, they are reluctant to give out personal information online, so they want to know how safe it is to tell the agency about their new address this way. http://www.usability.gov/how-to-and-tools/methods/scenarios.html

Storyboards

65

Customer  Journey  Mapping68

Web Mobile Phone Store

user  experience

hZp://www.adap`vepath.com/ideas/the-­‐anatomy-­‐of-­‐an-­‐experience-­‐map

http://adaptivepath.org/uploads/images/RailEurope_AdaptivePath_Touchpoints_V1.png

Setting  Design  Objectives

71

• Effect Goals • The expected effects of a project • Why a design work at all need to be made • What it should contribute with to the business and

to the users

Why-­‐why-­‐why

72

• Why do people object to traffic congestion?: Because it causes them irritation.

• Why does it cause irritation?: Because it delays one unexpectedly.

• Why are unexpected delays irritating?: Because they force one to alter one's plans.

• Why do road travellers have plans?: Because the car journey is not an end in itself…or is it? • http://deseng.ryerson.ca/dokuwiki/design:why_why_why

UX  Goals  –  How  should  it  be?

73

• The UX Analysis gave a list of qualities that should be emphasized or changed in the users’ experiences. E.g.: • Respect for the child • Empathy • Equality • Questioning of authorities • Respect for culture • Respect for nature • Curiosity • Breaking the norms

Objectives  Tree

74

Value

Desirable  quality  1

Desirable  quality  2

Desirable  quality  3

Quality  criterium  1

Quality  criterium  2

Quality  criterium  3

Quality  criterium  4

Quality  criterium  5

Quality  criterium  6

Business  Impact  Mapping

75

• What are the desired effects and values that the design should contribute to?

• Who are the stakeholders who can create this value and these effects?

• What are their goals? • How can the design contribute to them?

76

Fritt baserat på en beskrivning av Antrop.

Business  goals

Target  groups

Use  goals

Scenarios  and  func9ons

77

Webbplatsen

Effektmål 1 (E1)På webbplatsen framgår tydligt vilken roll myndigheten har.

Effektmål 2 (E2)På webbplatsen är det tydligt vilka stöd som finns för företagare och folk på regionala myndigheter.

Effektmål 3 (E3)Det ska vara enkelt att göra en ansökan och delta i program hos myndigheten.

Effektmål 4 (E4)Genom mål E1-E3 bidrar webbplatsen till att arbetsbelastningen på handläggare minskar.

© 2013 ANTROP

Effektmål 1 (E1) På webbplatsen framgår tydligt vilken roll Tillväxtverket har.

Effektmål 2 (E2) På webbplatsen är det tydligt vilka stöd som finns för företagare och regionala tillväxtansvariga.

Effektmål 3 (E3) Det ska vara enkelt att göra en ansökan om stöd och delta i program hos Tillväxtverket.

Effektmål 4 (E4) Genom mål E1-E3 bidrar webbplatsen till att arbetsbelastningen på handläggare och programansvariga minskar.

Tillvaxtverket.se E1, E2, E3 (E1), E2, (E3)

E2, (E3) (E1), E2, (E3)

24"

© 2013 ANTROP

Effektmål 1 (E1) På webbplatsen framgår tydligt vilken roll Tillväxtverket har.

Effektmål 2 (E2) På webbplatsen är det tydligt vilka stöd som finns för företagare och regionala tillväxtansvariga.

Effektmål 3 (E3) Det ska vara enkelt att göra en ansökan om stöd och delta i program hos Tillväxtverket.

Effektmål 4 (E4) Genom mål E1-E3 bidrar webbplatsen till att arbetsbelastningen på handläggare och programansvariga minskar.

Tillvaxtverket.se E1, E2, E3 (E1), E2, (E3)

E2, (E3) (E1), E2, (E3)

24"

© 2013 ANTROP

Effektmål 1 (E1) På webbplatsen framgår tydligt vilken roll Tillväxtverket har.

Effektmål 2 (E2) På webbplatsen är det tydligt vilka stöd som finns för företagare och regionala tillväxtansvariga.

Effektmål 3 (E3) Det ska vara enkelt att göra en ansökan om stöd och delta i program hos Tillväxtverket.

Effektmål 4 (E4) Genom mål E1-E3 bidrar webbplatsen till att arbetsbelastningen på handläggare och programansvariga minskar.

Tillvaxtverket.se E1, E2, E3 (E1), E2, (E3)

E2, (E3) (E1), E2, (E3)

24"© 2013 ANTROP

Effektmål 1 (E1) På webbplatsen framgår tydligt vilken roll Tillväxtverket har.

Effektmål 2 (E2) På webbplatsen är det tydligt vilka stöd som finns för företagare och regionala tillväxtansvariga.

Effektmål 3 (E3) Det ska vara enkelt att göra en ansökan om stöd och delta i program hos Tillväxtverket.

Effektmål 4 (E4) Genom mål E1-E3 bidrar webbplatsen till att arbetsbelastningen på handläggare och programansvariga minskar.

Tillvaxtverket.se E1, E2, E3 (E1), E2, (E3)

E2, (E3) (E1), E2, (E3)

24"

Fia, 37 årVD Mekoväst

Mia, 43 årStrateg för

företagarfrågor

Martin, 43 årChef på

utvecklings-kontor

Calle, 56 årProjektledare i

regional planering

E1, E2, E3 (E1), E2, (E3)

E2, (E3) (E1), E2, (E3)

Få en långsiktig överblick på hur myndighetens stödverksamhet ser ut

Få svar på områdesspecifika frågor

Nätverka med andra som är aktivainom samma område

Ta del av fakta och statistik

Bevaka vad som händer i regionen

Vara påläst om aktuella utlysningar och ansökningsomgångar

Tydliga svar

Vill veta vad som finns att söka förjust henne, i hennes situation

Vill bli guidad i hur hon ska göranär hon ska söka bidrag

Göra uppföljningar i detprojekt han leder

Ta del av nyheter i regionen

Få svar på projektspecifikafrågor från finansiärer

Building  an  impact  map  –  Adzic  (2012)

78

• As a <who?> • I want <what?> • So that I can <how?> • The <what?> can help me do that.

Design  is  a  learning  process

79

• Revise objectives • Specify objectives • Make them concrete as you learn more

www.liu.se

@mattiasarvola