CS 544 Discount Usability Engineeringcs544/Fall2002/544_heuristic_evaluation.pdf · 3 5 Discount...
Transcript of CS 544 Discount Usability Engineeringcs544/Fall2002/544_heuristic_evaluation.pdf · 3 5 Discount...
1
1
CS 544 Discount Usability Engineering
Heuristic Evaluation
Acknowledgement: Some of the material in these lectures is based on material prepared for similar courses by Saul Greenberg (University of Calgary), Ravin Balakrishnan (University of Toronto), James Landay(University of California at Berkeley), monica schraefel (University of Toronto), and Colin Ware (University of New Hampshire). Used with the permission of the respective original authors.
2
Usability Inspectionis the generic name for a set of methods based on having evaluators inspect or examine usability-related aspects of a user interfaceusability inspectors can be usability specialists, software development consultants with special expertise, end users with task/content knowledge, or other professionals4 ways to evaluate an interface:
– automatically (compute usability measures using evaluation software)
– empirically (usability testing with real users)– formally (using models to calculate usability measures)– informally (rule of thumb and general knowledge of evaluators)
usability inspection is an informal evaluation
2
3
Discount Usability EngineeringDeveloped by Nielson, based on using four
techniques:User and task observationScenariosSimplified thinking aloudHeuristic evaluation
4
Articulate:•who users are•their key tasks
User and task descriptions
Goals:
Methods:
Products:
Brainstorm designs
Task centered system design
Participatory design
User-centered design
Evaluate
Psychology of everyday things
User involvement
Representation & metaphors
low fidelity prototyping methods
Throw-away paper prototypes
Participatory interaction
Task scenario walk-through
Refined designs
Graphical screen design
Interface guidelines
Style guides
high fidelity prototyping methods
Testable prototypes
Usability testing
Heuristic evaluation
Completed designs
Alpha/beta systems orcomplete specification
Field testing
Interface Design and Usability Engineering
3
5
Discount usability engineeringCheap– no special labs or equipment needed– the more careful you are, the better it gets
Fast– on order of 1 day to apply– standard usability testing may take a week
Easy to use– can be taught in 2-4 hours
6
Heuristic EvaluationDeveloped by Jakob NielsenPrinciples used to “evaluate” a system for usability problemsBecoming very popular
user involvement not requiredcatches many design flaws
Small set (3-5) of evaluators examine UI– independently check for compliance with usability principles
(“heuristics”)– different evaluators will find different problems– evaluators only communicate afterwards
findings are then aggregatedCan perform on working UI or on sketches
4
7
Why multiple evaluators?Every evaluator doesn’t find every problemGood evaluators find both easy & hard ones
8
Heuristic Evaluation ProcessEvaluators go through UI several times– inspect various dialogue elements– compare with list of usability principles– consider other principles/results that come to mind
Usability principles– Nielsen’s “heuristics”– supplementary list of category-specific heuristics
competitive analysis & user testing of existing products
Use violations to redesign/fix problems
5
9
Neilson’s Heuristics (original)H1-1: Simple & natural dialogH1-2: Speak the users’ languageH1-3: Minimize users’ memory loadH1-4: ConsistencyH1-5: FeedbackH1-6: Clearly marked exitsH1-7: ShortcutsH1-8: Precise & constructive error messagesH1-9: Prevent errorsH1-10: Help and documentation
10
H2-1: Visibility of system statuskeep users informed about what is going on
– example: pay attention to response time 0.1 sec: no special indicators needed, why? 1.0 sec: user tends to lose track of data 10 sec: max. duration if user to stay focused on action for longer delays, use percent-done progress bars
searching database for matches
6
11
H2-1: Visibility of system statuskeep users informed about what is going on– Appropriate visible feedback
What did I select?
What mode am I in now?
How is the system
interpreting my actions?
12
H2-2: Match between system & real world
speak the users’ languagefollow real world conventions
(bad) example: Mac desktop– Dragging disk to trash
should delete it, not eject it!
7
13
H2-2: Match between system & real world
speak the users’ languagefollow real world conventions
e.g. withdrawing money from a bank machine
14
H2-3: User control & freedom“exits” for mistaken choices, undo, redodon’t force down fixed paths
How do I get out of this?
8
15
H2-3: User control & freedom“exits” for mistaken choices, undo, redodon’t force down fixed paths
Strategies:– Cancel button (for dialogs waiting for user input)– Universal Undo (can get back to previous state)– Interrupt (especially for lengthy operations)– Quit (for leaving the program at any time) – Defaults (for restoring a property sheet)
16
H2-4: Consistency & standardsConsistency of effects
– same words, commands, actions will always have the same effect in equivalent situations
predictability
Consistency of language and graphics– same info/controls in same location on all screens/dialog boxes
– forms follow boiler plate– same visual appearance across the system (e.g. widgets)
e.g. different scroll bars in a single window system!
Consistency of input– consistent syntax across complete system
9
17
H2-4: Consistency & standards
These are labels with a raised appearance.
Is it any surprise that people try and click on them?
18
H2-5: Error preventiontry to make errors impossiblemodern widgets: only “legal commands” selected, or “legal data” entered
Provide reasonableness checks on input data– on entering order for office supplies
5000 pencils is an unusually large order. Do you really want to order that many?
10
19
H2-6: Recognition rather than recallComputers good at remembering things, people aren’t!Promote recognition over recall
– menus, icons, choice dialog boxes vs command lines, field formats– relies on visibility of objects to the user (but less is more!)
20
H2-7: Flexibility and efficiency of useExperienced users should be able to perform frequently used operations quicklyStrategies:
– keyboard and mouse acceleratorsabbreviationscommand completionmenu shortcutsfunction keysdouble clicking vs menu selection
– type-ahead (entering input before the system is ready for it) – navigation jumps
e.g., going to window/location directly, and avoiding intermediate nodes
– history systems WWW: ~60% of pages are revisits
11
21
H2-7: Flexibility and efficiency of useKeyboard
accelerators for menus
Customizable toolbars andpalettes for
frequent actions
Split menu, with recently used fonts on top
Scrolling controls for page-sized increments
Double-click raises object-specific menu
Double-click raises toolbar
dialog box
22
H2-8: Aesthetic and minimalist designNo irrelevant information in dialoguesBad example:
12
23
H2-9: Help users recognize, diagnose, and recover from errors
error messages in plain languageprecisely indicate the problemconstructively suggest a solution
24
Errors we make
Mistakes– arise from conscious deliberations that lead to an
error instead of the correct solutionSlips– unconscious behaviour that gets misdirected en
route to satisfying goale.g. drive to store, end up in the office
– shows up frequently in skilled behaviourusually due to inattention
– often arises from similarities of actions
13
25
Types of slips
Capture error– frequently done activity takes charge instead of one intended
occurs when common and rarer actions have same initial sequence– change clothes for dinner and find oneself in bed (William James, 1890)– confirm saving of a file when you don’t want to delete it
I can’t believe I pressed
Yes...
26
Types of slips
Description error– intended action has much in common with others that are possible
usually occurs when right and wrong objects physically near each other– pour juice into bowl instead of glass – go jogging, come home, throw sweaty shirt in toilet instead of laundry basket– move file to trash instead of to folder
Loss of activation– forgetting what the goal is while undergoing the sequence of actions
start going to room and forget why you are going therenavigating menus/dialogs and can’t remember what you are looking forbut continue action to remember (or go back to beginning)!
Mode errors– people do actions in one mode thinking they are in another
refer to file that’s in a different directorylook for commands / menu options that are not relevant
14
27
Designing for slips
General rules– Prevent slips before they occur– Detect and correct slips when they do occur– User correction through feedback and undo
Examples– mode errors
have as few modes as possible (preferably none)make modes highly visible
– capture errorsinstead of confirmation, make actions undoableallows reconsideration of action by user
– e.g. Mac trash can can be opened and “deleted” file taken back out– loss of activation
if system knows goal, make it explicitif not, allow person to see path taken
– description errorsin icon-based interfaces, make sure icons are not too similar, check for reasonable input, etc.
28
Generic system responses for errors
General idea: Forcing functions– prevent / mitigate continuation of wrongful action
Gag– deals with errors by preventing the user from continuing
eg cannot get past login screen until correct password entered
Warn– warn people that an unusual situation is occurring– when overused, becomes an irritant
e.g., – audible bell – alert box
Do nothing– illegal action just doesn’t do anything– user must infer what happened
enter letter into a numeric-only field (key clicks ignored)put a file icon on top of another file icon (returns it to original position)
15
29
Generic system responses for errors
Self-correct– system guesses legal action and does it instead– but leads to a problem of trust
spelling correctorLets talk about it
– system initiates dialog with user to come up with solution to the problemcompile error brings up offending line in source code
Teach me– system asks user what the action was supposed to have meant– action then becomes a legal one
30
H2-10: Help and documentation
Help is not a replacement for bad design!
Simple systems:– walk up and use; minimal instructions
Most other systems:– feature rich– some users will want to become “experts” rather than “casual” users– intermediate users need reminding, plus a learning path
Many users do not read manuals– prefer to spend their time pursuing their task
Usually used when users are in some kind of panic, need immediate help– indicates need for online documentation, good search/lookup tools– online help can be specific to current context– paper manuals unavailable in many businesses!
e.g. single copy locked away in system administrator’s officeSometimes used for quick reference
– syntax of actions, possibilities...– list of shortcuts ...
16
31
Types of help
Tutorial and/or getting started manuals– short guides that people are likely to read when first obtaining their
systemsencourages exploration and getting to know the systemtries to get conceptual material across and essential syntax
– on-line “tours”, exercises, and demosdemonstrates very basic principles through working examples
Reference manuals– used mostly for detailed lookup by experts
rarely introduces conceptsthematically arranged
– on-line hypertextsearch / findtable of contentsindexcross-index
32
Types of help (cont.)Reminders– short reference cards
expert user who just wants to check factsnovice who wants to get overview of system’s capabilities
– keyboard templatesshortcuts/syntactic meanings of keys; recognition vs. recall; capabilities
– tooltipstext over graphical items indicates their meaning or purpose
17
33
Types of help (cont.)Context-sensitive help– system provides help on the interface component
the user is currently working withMacintosh “balloon help”Microsoft “What’s this” help
– brief help explaining whatever the user is pointing at on the screen
Wizards– walks user through typical tasks– but dangerous if user gets stuck– Remember the MS “paperclip”?
34
How to perform evaluationAt least two passes for each evaluator
– first to get feel for flow and scope of system– second to focus on specific elements
If system is walk-up-and-use or evaluators are domain experts, no assistance needed
– otherwise might supply evaluators with scenarios
Each evaluator produces list of problems– explain why with reference to heuristic or other information– be specific and list each problem separately
18
35
How to perform evaluation (cont.)Why separate listings for each violation?– risk of repeating problematic aspect– may not be possible to fix all problems
Where problems may be found– single location in UI– two or more locations that need to be compared– problem with overall structure of UI– something that is missing
hard w/ paper prototypes so work extra hard on thosenote: sometimes features are implied by design docs and just haven’t been “implemented” – relax on those
36
Severity ratingsUsed to allocate resources to fix problems Estimates of need for more usability effortsCombination of– frequency– impact– persistence (one time or repeating)
Should be calculated after all evals. are inShould be done independently by all judges
19
37
Severity ratings (cont).0 - don’t agree that this is a usability problem1 - cosmetic problem 2 - minor usability problem3 - major usability problem; important to fix4 - usability catastrophe; imperative to fix
38
SummaryAdvantages
– the “minimalist” approacha few general guidelines can correct for the majority of usability problemseasily remembered, easily applied with modest effort
– discount usability engineeringcheap and fast way to inspect a systemcan be done by usability experts, double experts, and end users
Problems:– principles are more or less at the motherhood level
can’t be treated as a simple checklistsubtleties involved in their use
– doesn’t necessarily predict user/customers overall satisfaction
– may not have the same “credibility” as user test data (one solution to this is to include the design team and developers in the usability evaluation, as in a “pluralistic walkthrough”)
20
39
SummaryResearch result:
– between 4-5 evaluators appear to be sufficient in most cases to identify 80% of total usability problems
– user testing and usability inspection have a large degree of non-overlap in the usability problems they find (i.e., it pays to use both methods)
Cost-benefit:– usability engineering activities are often difficult to justify and
carry out in a timely way, but many activities can be done quickly and cheaply, and produce useful results
– usability inspection turns less on what is “correct” than on what can be done within development constraints
– ultimate trade-off may be between doing no usability assessment and doing some kind
40
You now know10 Heuristics for evaluating designs:
– H2-1: Visibility of system status – H2-2: Match between system & real world– H2-3: User control & freedom – H2-4: Consistency & standards– H2-5: Error prevention– H2-6: Recognition rather than recall– H2-7: Flexibility and efficiency of use– H2-8: Aesthetic and minimalist design– H2-9: Help users recognize, diagnose, and recover from errors – H2-10: Help and documentation
Principles can be used to systematically inspect the interface for usability problems
21
41
Readings and ReferencesBGBG 80-86 (Evaluating Systems and Their User Interfaces)
BGBG 170-181 (Usability Inspection Methods)
Optional:Neilsen, J. (1993). Usability Engineering. Read Chapter 5: Usability Heuristics, 115 – 163.