Inspection Methods

Inspection methods

• Heuristic evaluation• Guidelines review• Consistency inspections• Standards inspections• Features inspection• Cognitive walk-throughs• Pluralistic walk-throughs• Competitive evaluation

Heuristics vs. guidelines & standards

• Heuristics– General (e.g., “rely on recognition rather than

recall”)– Informal– Often develop one’s own

• Guidelines– More specific (“include home link on every

page”)– Often organization-wide

• Standards– Closer to force of law

Concerns

• Justifiability of critiques: what is the basis (other than taste, opinion)?– Empirical: users; research; competitors– Expert opinion: guidelines, standards

• Communicability: what exactly is wrong and what needs to be done?

• Severity of violation: priority? Cost/benefit?

• Accountability: how will you know when the problem has been remedied?

Components of a walk-thru

• One or more task scenarios• Explicit assumptions about the user

population and contexts of use– personas

• Sequence of actions a user is likely to perform to complete the task– scenarios

• Prototype of some sort– Incl. competitor site

• MAY include heuristics, guidelines, other criteria

Walkthroughs

• Cognitive– Taking a systematic look at any product with

an eye on ease of learning by inspection.• Rationale: acceptability depends on first experience

• General: Using prototypes, task flows, personas, and scenarios– Walk in your users' shoes through your

application– Try out parts of the design, following a task

flow or scenario – Look for problems

Pluralistic walk-through

• Involves several different groups, typically:– Users– Product developers– Usability experts

• They walk through one or more scenarios separately and then together using a prototype

Benefits and limits

• Short and fairly simple• Allows developers to hear the concerns

of users with the system directly, early enough to do some good

• Questions of validity given the constrained setting and tasks

• Scenarios don’t readily reflect the full gamut of possible uses and users

Competitive usability study

• Purpose: gather insights from related sites

• Look for both problems and good ideas• Understand the context within which

users will be working– Other choices– What they are used to

• What is salient for users: whatever was sacrificed often becomes important in the next round

• Caution: easy to focus on trivia

How to do competitor study

• Identify competitors• Perform usability inspection by

developers or usability specialists• Perform usability testing by users

– With directed tasks– With their own tasks

Selecting sites for competitive usability study

• Sites that have good reputations• Sites that have interesting features or

designs• The market leaders• Sites your users may be familiar with• Sites that have bad reputations, for

mistakes to avoid• Sites that are considered cutting-edge• Ask users!

What to look for

• Targeted users• The user goals, tasks this site supports• Content, functionality, navigation, design • Things a site does well

– Why and how?

• Things it does poorly– Why and how?

• Ideas to adopt• Things to avoid

Standards

Types of standards

• Use of the product in context of use– Including ergonomics

• Interface and interaction• Development process• Capability of organization to apply

user-centered design

ISO 13407: human centered design processes for interactive systems

– Understand and specify the context of use

– Specify user and org’l requirements– Produce design solutions– Evaluate designs against

requirements

Guidelines

Guidelines

• Often specific to an organization or type of organization or application

• Embody accumulated wisdom about good design

• Ensure consistency of look, functionality, information architecture, and so forth, e.g. within an organization

• E.g., IBM Web design guidelines http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/572

Specialized Heuristics: e-Commerce and Order Forms

From http://www.weinschenk.com/tools/online_checklist.asp

• Shows total cost • Shows itemized costs • Shows product names and/or descriptions • Allows the user to change the quantity easily• Provides an option to save an order and complete it later • Provides details on any other charges on the order • Provides details on shipping options and charges • Provides shortcuts for repeat visitors to make transactions faster • Allows users to easily move from the order form to shopping and back again • Provides security information • Provides users with an alternate offline way of ordering• Allows users to view and/or change previous orders • Does not require users to register before a purchase

E-commerce IIhttp://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/611

• Customer support: Supporting users before, during, and after a purchase.  

• Trust: Establishing trustworthiness. • Product Navigation: Enabling users to browse

products easily. • Product Information: Providing the product

information that users want, need, and expect. • Purchase transaction: Providing easy means for

users to purchase products.

Customer Support(IBM guidelines)

• Provide contact information on every page• Provide assistance when users have forgotten their

passwords• Provide clear and informative error messages• Address users' frequently asked questions• Provide simple definitions and explanations of

important terms• Provide product selection assistance• Provide assistance to guide users through multiple

step processes

Heuristic evaluation: plusses and minuses

• Benefits– Low resource requirements– Usually find many problems fairly quickly– Easy to repeat in iterative design– Easy to communicate– Usually easy to get agreement on a basic set of heuristics– Face validity

• Limits– Can be superficial– Focuses on easily-seen problems; harder to find more subtle problems

associated with in-depth use, repeated use– Can be deceptive – assumption that evaluation has been more complete

and thorough than it has been– How similar to users are experts? How expert are users?– How appropriate are the heuristics to THIS site?– Tends toward a short list of heuristics– Trade-offs among heuristics, the fixes needed?

Some key points

• We need to differentiate among official standards, how people generally do things, and expert opinion

• Usability in a changing environment: what people are used to, their technology and expectations, are continually evolving

• Heuristics need to be customized to goals, context

• Most guidelines are solutions to problems; have to ask what is the underlying rationale, goal

• Trade-offs among different goals, heuristics often have to be mad

Heuristics - observations

• Heuristics shape what we see• What we think the problems may be

help shape decisions about heuristics• As a practical matter, tend to focus on

problems, not what a site/system does well

• CONTENT is not addressed by most heuristics and guidelines