A brief overview of ux research
-
Upload
panagiotis-zaharias -
Category
Design
-
view
329 -
download
1
description
Transcript of A brief overview of ux research
A brief intro to UX Research
Dr. Panagiotis Zaharias
13/5/2014 @ CoLab Athens
The UX Mantra
To measure is to know.
If you can’t measure it,
you can’t improve it…
Overview of the talk
Examples of UX evaluation studies
Current trends in UX evaluation
UX Research: Main Concepts
Main Concepts in UX Research
First things first
The Ancestor: Human Computer Interaction
Human – Computer Interaction, is “a discipline concerned
with the design, evaluation and implementation of interactive
computing systems for human use and the study of major
phenomena surrounding them’’.
User friendly? Usable? …
Once upon a time we were talking about “user friendly” systems…
… so general and vague
Then usability emerged and was originally derived from the term ‘‘user friendly’’
… but usability is still a general and vague term
Many definitions and different approaches to how usability should be measured
What is usability anyway? The ISO definition (9241-11)
Usability is defined as: ‘‘the extent to which a product can be
used by specified users to achieve specified goals with
effectiveness; the extent to which the intended goals of use are
achieved,
efficiency; the resources that have to be expended to achieve the
intended goals and
satisfaction; the extent to which the user finds the use of the
product acceptable,
in a specified context of use’’.
Examples of usability measures (1)
Measures of effectiveness: Task completion (number of correct tasks, number of tasks
where users failed to finish within a set time, number of tasks where users gave up)
Accuracy measures (quantify the number of errors users make during the process of completing tasks)
Recall measures (how much information users can recall after the use of an interface)
Quality of outcome (attempts to measure the outcome of the tasks, measures of understanding e.g. tests of what has been learned from an e-learning system)
Examples of usability measures (2)
Measures of efficiency Time (refers to measures of how long users take to
complete tasks)
Input rate (e.g. words per minute)
Mental effort (mental resources spend on interaction, e.g. heart rate variability, subjective time estimation)
Usage patterns (number of times a certain action has been performed, how much information users access when solving tasks, deviation from the optimal solution etc.)
Examples of usability measures (3)
Measures of satisfaction Preference measures capture which interface users prefer
using
Specific attitudes towards the interface (liking, fun, annoyance etc.), toward the content of interface (quality of information, organization of information etc.)
Perception of outcomes (users’ assessment of their performance, users’ perception of learning, users’ confidence in the solution to tasks)
From Usability to User Experience (1)
UX - A relatively new field - Lack of unique definition…
But let’s take for example the ISO definition on UX:
“A person's perceptions and responses that result from the use
or anticipated use of a product, system or service”
From Usability to User Experience (2)
Usability
• Focused on optimizing task efficiency
• Removing problems in the user interface
• Studying interactive products mainly used for work
UX
• Dynamic context-dependent and subjective
• Focuses also on hedonic qualities of use (identification, stimulation, attractiveness etc.)
Current trends in UX evaluation
As we shift to User Experience…
Challenge#1: Need for new approaches and methods
Existing methods are somewhat unable -it is claimed- to
capture experience
Challenge#2: The choice of UX evaluation method
Qualitative or quantitative methods?
Only few studies combine qualitative and quantitative methods
What Dimensions of Experience Are
Studied?
Usability
• Effectiveness
• Efficiency
• Satisfaction
UX
• Emotions and affect
• Enjoyment
• Aesthetics
• And many more…
What Dimensions of Experience Are
Studied?
Source: Bargas- Avila and Hornbaek, 2012
How do we measure UX?
What are the methods applied in UX
Qualitative (50 %), quantitative (33%), combination (17%)
How UX data are collected?
Questionnaires are still the dominant UX evaluation method
(53%).
Source: Bargas- Avila and Hornbaek, 2012
Popular UX evaluation methods
Source: Bargas- Avila and Hornbaek, 2012
When to use which UX evaluation methods
Decision
Qualitative vs.
Quantitative
Attitudinal vs.
Behavioral
Context of system/
product use
Source: http://www.nngroup.com/articles/which-ux-research-methods/
The Attitudinal vs. Behavioral Dimension
to understand, measure, or inform change of people's stated beliefs
e.g. self-reported methods such as surveys, interviews, focus groups, etc.
Attitudinal to understand "what people do" with minimal interference from the method itself
e.g. A/B testing, observation and video recording, first click testing, eyetracking, etc.
Behavioral
Source: http://www.nngroup.com/articles/which-ux-research-methods/
The Qualitative vs. Quantitative Dimension
- the data is usually being gathered directly
- they are much better suited for answering question about why or how to fix a problem
Qualitative - the data is gathered indirectly, through an instrument, such as a survey or a web server log etc.
- they do a much better job answering how many and how much type of questions.
Quantitative
Source: http://www.nngroup.com/articles/which-ux-research-methods/
The Context of Use Dimension
This dimension can be described by the following options:
Natural or near-natural use of the system/product
Scripted use of the system/product
Not using the system/product during the study
A hybrid of the above
Source: http://www.nngroup.com/articles/which-ux-research-methods/
What about the use of UX evaluation
methods across the development phases ?
UX evaluation methods across development
phases
Product Development Phase
Strategize Optimize Assess
Goal: Inspire, explore and
choose new
directions and
opportunities
Inform and optimize
designs in order to
reduce risk and
improve usability
Measure product
performance against
itself or its
competition
Approach: Qualitative and
Quantitative
Mainly Qualitative
(formative)
Mainly Quantitative
(summative)
Typical
methods:
Ethnographic field
studies, focus groups,
diary studies,
surveys, data mining
or analytics etc.
e.g. cardsorting, field
studies, observation,
paper prototype and
usability studies, etc.
e.g. Usability
benchmarking,
surveys, A/B testing,
etc.
Source: http://www.nngroup.com/articles/which-ux-research-methods/
Some examples from UX research
studies
Walls of Nicosia The context:
"The Walls of Nicosia” is an interactive application that runs on a multi-touch table at the Leventeio Museum, Nicosia
Students from elementary schools as museum visitors
Goal: To conduct a comparative
summative evaluation
Methods used: Multiple choice knowledge tests
UX questionnaires (FunToolkit)
Observation
Indicative results: Mainly quantitative analysis (pre and post test analysis, t-tests)
UX questionnaire for children: the
FunToolkit (Read, 2007)
The Again-Again table
The Smileyometer
Source: http://hrast.pef.uni-lj.si/docs/research/Serious%20games/evaluation_paper_Janet.pdf
Long term UX evaluation of League of
Legends (1)
The context:
The well known MMOG “League of Legends”
Players were students at TEI of Kavala
Goal: to identify how UX of players
changed over time
Methods used:
An adaptation of UX Curve
Self reported method by drawing curves that depict changes in
UX over time
Long term UX evaluation of League of
Legends (2)
Indicative results:
Reason Categories Positive Negative
General UX 43 14
Ease of use 41 26
Immersion 43 10
Social interaction 38 12
Engagement 32 10
Total 197 72
General UX Curve for each player
Number of reasons for UX dimensions
Heuristic Evaluation of LMS
The context:
A well known cloud-based Learning Management System platform
Goal: to identify usability problems and possible areas for UX improvement
Methods used:
Heuristic evaluation
Heuristic evaluation protocol adapted by Mehlenbacher et al. (2005)
Indicative results: An evaluation report that contains the identified usability and
UX problems, their severity, and relevant redesign suggestions
Typical structure of the report of heuristic
evaluation
A numeric identifier of the problem A short description of the problem (and likely difficulties
for the user) Specific context (defined as location of the problem in the
interface): we identified this generically in terms of affecting the: Structure of the system, including layout, navigation, and basic help
functions such as print, help, etc. Content, including actual content for study, exploration, examples,
exercises, and so on Global issues, i.e., affecting both structure and learning content
The heuristic(s) used The severity rate: severe, moderate, and minor
First Click testing with mockups
The context:
A project for the design of new MOOC interfaces
Goal: to create, inform and optimize MOOC interface
design
Methods used:
Interviews and first click testing with users
First Click testing - indicative results
First Click testing - indicative results
0
2
4
6
8
10
12
Ραβδόγραμμα εργασιών
Σωστές εργασίες Λανθασμένες Εργασίες
Some UX resources:
World’s premier magazine for both academics and industry:
http://interactions.acm.org/
A comprehensive list of UX methods:
http://www.allaboutux.org/
An indicative list of UX tools:
http://www.smashingmagazine.com/2011/10/20/comprehensive
-review-usability-user-experience-testing-tools/
Other sources on the web:
http://uxmag.com/
http://www.usability.gov/
http://www.uxmatters.com/index.php
Key takeaways:
You can’t afford NOT to evaluate UX
Always use a combination of methods
Don’t forget Usability
Do it properly…
Thank you!
Let’s keep in touch:
Web: about.me/panagiotis_zaharias
Email: [email protected]
LinkedIn: gr.linkedin.com/pub/panagiotis-zaharias/0/bb6/a91
Twitter: twitter.com/pan_sugarman