Amela Karahasanović - Universitetet i oslo · Amela Karahasanović Research Scientist, SINTEF IKT...
Transcript of Amela Karahasanović - Universitetet i oslo · Amela Karahasanović Research Scientist, SINTEF IKT...
Amela Karahasanović
Research Scientist, SINTEF IKT
Adjunct Associate Professor, DESIGN
www.sintef.no
Sintef IKT, Forskningsveien 1
http://heim.ifi.uio.no/~amela/
•User modeling
•Usability and user
experience
•Methods for empirical
evaluation in HCI and SE
•Software maintenance
•Program comprehension
Technology for a better society
•Largest Scandinavian independent research organisation
•More than 2100 employees from 67 countries
•Generated knowledge for 2.8 billions of NOK in 2010
•Eight research divisions
•About 16% of our turnover is derived from international projects
2
HCI research group
– Psychology
– Software Engineering
– Media science
3
Wednesday 19th September 10:15 – 12:00
http://gemini-centre-user-experience.origo.no/
– Bobble experiment (3D technology)
– Living Lab studies
– Collaboration tool
4
Plan for today
Chapter 10 + some examples
• What is usability testing
• Usability testing versus traditional research
• Types of usability testing
• User-based testing
• Questions for discussion and homework
5
What is usability testing?
• Involving representative users, representative tasks, representative environments
– Testing paper prototypes
– Wizard of Oz
– Testing working version before it is released
– Testing working system
– Different devices (smart phones, laptops…)
6
Goal of usability testing
• Improve the quality of an interface by finding flaws in it
• Flaws that cause problems for the majority of people
7
8
Usability versus research?
• Approach similar as one used in classic research
• Different goals
• Usability testing is involved in building a successful product (resources, time, trade-offs)
– Practical, large impact
– Optimize interface after each iteration
• Usability study – Research study - Research on usability methods
9
Usability engineering
Activities aiming to improve the ease of use of an interface
• Expert-based testing (usability inspection)
• Automated testing (usability inspection)
• User-based testing (usability testing)
10
Expert-based testing
• Structured inspections done by interface experts • Before tests with users • Confusing wording, inconsistent layout, obvious flaws • Heuristic review
– Compare interface with the rules
• Consistency inspections – Series of screens or web pages inspected
• Cognitive walkthrough – Experts perform the tasks (high-frequency and
important/seldom)
• Guidelines review – Web Content Accessibility Guidelines
11
Automated usability testing
• Software that compare interface with the guidelines
• Produce report and/or fix the code
• Manual check often needed
• <alt> tag but not if the text is appropriate
• Number of fonts, avg. font size, deepest level of a menu
• RAMP, InFocus
12
User-based testing
– Select representative users
– Select the setting
– Decide what tasks users should perform
– Decide what type of data to collect
– Before the test session (informed consent, etc.)
– During the test session
– Debriefing after the session
13
Questions – Part I
Go to the alarm app on you phone
– Difference between a summative and a formative usability test of this app
– Should we first test it with users or with experts? What do you think? Why?
– What are the differences between a usability test and an experiment with this app?
14
Stages
– Users perform tasks (early or later in the development)
– Formative testing • Low-level fidelity prototypes
• How the user perceives an interface component?
• Low-cost of paper prototypes, users comfortable to criticize
– Summative testing • Evaluate the effectiveness of specific design choices
• High fidelity prototype
– Validation test • Before release, compare to benchmarks
– Formal approach 15
Usability evaluation Talhonia
time
16
Usability evaluation Talhonia – cont.
17
How many users?
– 5 users will find approximately 80% of problems
– 7 for small projects, 15 for medium-large projects
– Goal to find the major flows , that will cause the most problems and must be fixed
– How many users can we afford? How many users can we get? How many users do we have time for?
18
Location
– Lab
– Portable usability lab
– Remote studies (time, place)
19
Tasks
• Clear; no need for further explanation
• Tested
• One clear answer – way to do things
• Tasks that are performed often
• Critical tasks – logging
• No private, financial information
• Must be clear how to go to the next task
20
Oppgave 4: Du er hjemme og du ser NRK Nett-TV på din PC ( http://www.nrk.no/nett-tv ).
Se på episoden 10:12 av Matlyst og svar følgende spørsmål:
• Hva heter kvinnen som er kledd i lilla genser og brunt forkle?
• Når en potet frøs om vinteren, hva ble den da?
21
Measurement
– Task performance
– Time performance
– User satisfaction (validated survey tool)
– Average time to recover from an error
– Time spent using help
– Number of visits to the search feature
– Time spent on specific web page
– Typing speed
– Qualitative data
0
5
10
15
20
25
30
35
40
45
1 3 5 6 7 10 11 12 13 17 18 20 21 23 24 25 26
Unified Library Application - Quit
Unified Library Application
Title Information
ReturnItem
Reserve Title
Lend Item
Insert Title Window
Find Title
Compilation
Subject_id 15 Task 2
Sum of Seconds
Minutes
Visited
22
Starten av dagen: • Sjekke at alt er på plass i laben; gavekort, utstyr, godteri, brus, batterier, pen og papir osv. • Skrive ut eksperimentet material som skal deles til deltagerne • Skrive ut tlf.nr. til de som kommer (mail fra Nordstat) • Informere resepsjonen at vi venter deltakere. Hvis deltaker kommer veldig tidlig, be resepsjonisten
henvise dem til sofa i resepsjonen. • Logge seg på PC og teste alt virker (ingen updates skal dukke opp i løpet av eksperimentet) • Gå inn på facebook og twitter og sjekk at ”thomtor” er logget ut • Heng IKKE FORSTYR på døra • Ha alltid mobilen med deg Eksperimentet (før deltakeren kommer) • Start SINTEF verktøyet (r2d2exp.jar). Det kan ta litt tid – IKKE KLIKK FLERE GANGER! • Start opp camtasia studio (dobbelt klikk) • Start opp uLog (høyre klikk på ikonet i ”system tray” og start) • Eksperiment (hent deltaker) • Ta med deltageren ned; hilse; introduksjon • Konfidensialitet skjema signeres • Husk å sette mobilen på vibrasjon • Start opptak i camtasia studio (høyreklikk, klikk ‘record the screen’, klikk record button) • …
23
24
Making sense of the data
– Write up the results and help influence the design of the specific interface
– Presentation to developers and managers
– Report should • Include all flaws
• Priorities
• For each flaw: describe the problem, present the data, priority, suggest a fix, estimate time/efforts for the fix
– Report structure • How you did usability testing and how you prepared
• What happened during the testing
• The implications and recomendations
25
Questions – Part II Alarm app usability test
• What is important to remind participants before the usability test?
• What is the "think aloud" protocol? Should it be used in summative or formative studies?
• Properties of good tasks?
26
Assignment
Make a PLAN for a summative evaluation of the ‘Contacts’ functionality on your mobile phone in a usability lab with 10 participants.
27