Experience Research Best Practices
-
Upload
dan-berlin -
Category
Design
-
view
106 -
download
3
description
Transcript of Experience Research Best Practices
September 19, 2013
Dan BerlinExperience Research [email protected]@banderlin
Mad*Pow
EXPERIENCE RESEARCH BEST PRACTICES
Hi! I’m Dan Berlin @banderlin
BA in psychology from Brandeis U. Studied visual space perception
Seven years in technical support Sat as a participant for a usability study for a product I was working on
Realized that user experience (UX) work is the perfect combination of computers and psychology
Went to Bentley U. to earn an MBA and MS in Human Factors in Information Design
Two years at an interactive agency performing usability and neuromarketing research Then did some freelance UX consulting for about a year
Am now an Experience Research Director at Mad*Pow, an experience design agency based out of Portsmouth, NH
My passion is for research methodology and finding new ways to elicit data from participants
2
About this Presentation
Understanding your research goals Formative and evaluative studies
How to choose the right method(s) Methods chart
Gathering qualitative data Taking notes and organizing findings
3
UNDERSTANDING YOURRESEARCH GOALS
4
Understanding Your Research Goals
5
Preparing to prepare for your study
TechnologyUser
Needs
BusinessGoals
Your overall goal:
To uncover actionable business and design insights via user data
Start by understanding the business’ goals What is the overall goal of the interface? What are the interactions that drive the business? What are the important calls to action? Who are the target audiences? What do they want to know about their users? What do they think they already know about their users
How did they learn this?
Understanding Your Research Goals
6
Preparing to prepare for your study
Actionable = the data should indicate what exactly needs to change or be implemented in the interface to align it with users’ expectations and needs
This means you need to start with actionable (and attainable) goals, such as: How do perform a certain interaction today? Does the proposed IA align with user expectations? Are users able to complete a critical transaction? Do users understand the pricing schedule?
Goals will depend on the type of study Formative
Performed at the beginning of a project to learn how the project should proceed
Evaluative Performed during and towards the end of projects to determine if the design aligns with user expectations
Formative
In-Person Usability
Interviews
Focus Groups
Collaging
Ethnography
Surveys
Diary Studies
Card-Sorting
7
Evaluative
In-Person Usability
Remote Usability
Unmoderated Usability
Desirability Testing
Eye-Tracking
Understanding Your Research GoalsStudy Types
HOW TO CHOOSE THERIGHT METHOD(S)
8
How to Choose the Right Method(s)
Make sure your research goals are clear Know what you want to learn
Know what you will do with the information you gather
Know what decisions the business needs to make
Know your constraints Timeline?
Resources?
Budget?
Access to Users?
Create a Methods Chart The answers will become clear
9
Know Goals and Constraints
How to Choose the Right Method(s)
10
Method Pros Cons Candidate
Method 1 • List the advantages for this method for this specific project
• List the disadvantages for this method for this specific project
Yes / No – Would this method be a good candidate for this specific project
Method 2
Method 3
Etc…
Methods Chart
How to Choose the Right Method(s)
Example Project Goals: Have 3 concepts for behavior change application. Which one is the best to
develop for maximum global appeal?
Client wants answers as quickly as possible
Client wants large numbers to provide confidence behind decision
Client wants data collection in 5 countries around the world
Designers want to know why or why not users chose each concept, to provide additional design direction
User population – adults who own a smart phone and want to get healthier
11
Methods Chart
Methods Chart Example
12
Method Pros Cons Candidate
Interviews • Good for capturing motivations for behavior change
• Can get good qualitative details on why users prefer each concept or not
• Good to explore issues to fine tune survey questions
• Can be done via phone (get broad geographic sample)
• Can be done quickly
• Small numbers – client wants large numbers
Y – would be good as a qualitative method to pair with a larger quantitative method
Focus Groups • Good for qualitative information gathering
• Could generate some interesting conversations about behavior change motivations
• Wouldn’t get as much detail as interviews
• Concern about group think when evaluating concepts
• Concern about not sharing details of personal goals in front of others
N – interviews would be better for qualitative
Survey • Good for large numbers• Easily replicated across
different countries/languages
• Can be done online for broad geographic distribution
• Can be done quickly
• Unclear what exact questions to ask
• Doesn’t provide detailed insights into qualitative topics
Y – good paired with qualitative method
How to Choose the Right Method(s)
Example Research Plan: Conduct telephone interviews in US with 12-15 participants
Create online survey in US for 300 participants
Use international research partners to conduct 12-15 interviews in 4 countries
Use international research partners to conduct survey for 300 participants in same 4 countries
Why this approach works: Interviews provide input into questions for survey
Survey provides large numbers; interviews provide in-depth insight
Both are commonly used methods and easy to replicate in different countries
Both methods can be done quickly
13
Methods Chart
GATHERING QUALITATIVE DATA
14
Gathering Qualitative Data
15
Usability Task Creation
Good usability study tasks are:
Non-leading: don’t give away the answer
Single tasks: don’t have users do two things in succession
Realistic: don’t have users do things they would not normally do
Self-explanatory: don’t have multiple sentences explaining the task
Achievable: always document what constitutes a “pass” for the task
Gathering Qualitative Data
16
Moderator’s Guide
A good study guide: Conveys the study goals and methodology to clients Serves as a quick reference to the moderator during the study Provides the moderator with a template to take hand-written notes, with ample space to
do so
The typical moderator’s guide contains:
Research method: a short paragraph explaining the usability study Study goals: a short bulleted list of the study goals Introduction: the moderator’s opening spiel when explaining the study to the participant Background questions: Typically demographic or product usage questions Tasks: The task, pass condition(s), notes for the moderator, and post-task questions Follow-up questions: The questions to be asked after all the tasks are complete
17
Gathering Qualitative Data
18
Notes Grid
Proper planning for taking notes is very underrated
The organization and thoroughness of your notes will dictate the ease with which you will create the final report
Organized, complete notes = easy reporting
Disorganized, incomplete notes = back to the video you go (ewww!)
Make your notes grid once your study & moderator’s guides are complete Give each question and subquestion its own row
Put each participant in a new column (or vice-versa, if you like)
DO NOT put each participant/task in a new worksheet
Use data validation for quantitative data Task ease ratings, task success, multiple choice questions, etc.
Always include an extra “Why?” cell for data validated cells (to capture qualitative data related to the question)
19
• Do not include the participant name in your notes grid
• Include a date/time cell to best align with the videos
• Visually separate sections of the study
• Fill data validated cells with a light color
• Hide columns when you have moved on to the next participant
Gathering Qualitative DataNotes Grid
20
Gathering Qualitative DataFindings Sheet
Scan the notes grid to determine findings Put these in a spreadsheet and assign and priority and category
In Conclusion
Set actionable and attainable research goals Know where you are in the project
Use a methods chart to weigh factors and choose a method Time, participants, goals, budget, etc.
Proper documentation = easier qualitative analysis Study guide notes grid findings spreadsheet report
21