Usability Tests
As presented in ProductCamp Boston 2015
Liron Zighelnic
Start discovering fashion you will LOVE!
Agenda – or: do you want to stay?
• What is usability• Why usability is important?• When and what to test?• How to test?• How many participants?• Facilitating• Metrics• 5 lessons about mobile usability test• Live demonstration
Start discovering fashion you will LOVE!
About me
• The Co-founder and CEO of CurtainApp • An MBA candidate at MIT with a passion for business and technology.• Have ten years of experience in tech, including SW development,
technology management and product management, in a variety of industries including military, intelligence, mobile, venture capital and consulting.• LinkedIn: linkedin.com/in/lironzighelnic • Email: [email protected]
Start discovering fashion you will LOVE!
About CurtainApp
• CurtainApp is an intelligent mobile app that learns your taste and gives you personal fashion recommendations, making shopping fun and efficient
• Register to the beta at: www.curtainapp.com • Join us on Facebook: facebook.com/CurtainApp • Follow us on Twitter: twitter.com/thecurtainapp
Start discovering fashion you will LOVE!
What is usability?
“Usability really just means making sure that something works well: that a person of average (or even below average) ability and experience can use the thing – whether it’s a web site, remote control, or revolving door – for its intended purpose without getting hopelessly frustrated.” Don’t Make Me Think by Steve Krug
Start discovering fashion you will LOVE!
Why usability is important?
• Real data
• Can save time and money
• In today’s world users expect products to have great UX (and
not only in B2C)
• Buy-in from team and stakeholders
Start discovering fashion you will LOVE!
When to test?
• Test early and often – identify problems before they get
designed/coded/released
• Test at each stage of the development
• In Agile environment test every sprint
Start discovering fashion you will LOVE!
When & what to test? & CurtainApp examples
Research
New ideas - papers
Design
• Paper prototypes• Wireframes (e.g.,
Balsamic)• Mockups (e.g.,
proto.io)
Development
• HTMS/mobile/code prototype• A/B testing• Navigation• Interactions
Live
• Live product• Test everything,
all the time
When
What
Start discovering fashion you will LOVE!
How to test?
Facilitating
Analyzing
Planning
Planning• What? (questions and tasks) • Who (based on persona & how
to recruit participants• Where? (lab vs. natural env.)
Facilitating• Explain• Moderate• Ask participants to ‘think aloud’After the session ends:• Listen and observe• Thank your participants!• Answer any questions they may have • Ask for feedback
Analyzing• Qualitative • Quantitative
Start discovering fashion you will LOVE!
5 participants will help you find 80% of the problems; 10 will help you find >90%
Source: JAKOB NIELSEN
Start discovering fashion you will LOVE!
Facilitating – cont.
• Observe real target users using the system• Observe them complete typical tasks• Collect qualitative and quantitative data on participants’ performance• ‘Think aloud’ – ask the user to share his/her thoughts and feelings
while operating the product • Important: first test behavior and only than opinion • Additional considerations: • Testing one user at a time vs. a few• A/B testing
Start discovering fashion you will LOVE!
Questions – step by step
1. General questions
• What is your impression?
• What do you see?• What do you expect? • What do you think that
the system does?
• What are you currently thinking?
• What are you looking for? What do you want to do?
• What do you think will happen next/will happen when you will press this?
2. Task-based questions
• Ask for general feedback
• Ask for specific feedback
3. Post test
Start discovering fashion you will LOVE!
Metrics
• Completion rate - task success
• Task time & efficiency (# of steps)
• Errors
• Satisfactions / how does the user feel? (SO important)
Start discovering fashion you will LOVE!
5 lessons about mobile usability test
• Don’t forget the size of the screen (for paper tests and wireframes)• Android & iPhone• Different type of personas• Test different devices (as many as possible) • If you use a freemium model – do people enjoy/find value in the free
features? Do they have a reason to convert to paid usage?
Start discovering fashion you will LOVE!
Let’s practice! – live demonstration
Start discovering fashion you will LOVE!
References
• Don’t Make Me Think - Steve Krug• Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of
the finding of usability problems,"Proceedings of ACM INTERCHI'93 Conference (Amsterdam, The Netherlands, 24-29 April 1993), pp. 206-213.• Usability Professional Association• Usability.gov
Top Related