Usability and Evaluation

How to get the most out of walkthroughs and playtesting

Chelsea HoweActionXL 11/03/09

huge thanks to Dan Cosley, HCI 3450

Why evaluate?

Find bugs earlier Address design issues Increase user enjoyment (more fun) Objective look at features and failures Prioritize the big To Do list

Makes your game BETTER

How to evaluate? Depends on where you are Different stages have different requisites What you’re looking for changes Stage

Concepting Prototyping Internal Playtest External Playtest BETA+

Method Cognitive Walkthrough Cog/Phys Walkthrough Heuristic Eval/Nielsen HE, Surveys, Observation Surveys, Observation, Forum

Tasks vs. Free play Giving evaluators “tasks” is useful in

forcing people to explore an interface Guided evaluation can focus people’s

attention where you want it Careful, tasks can be biased Hardest issue is choosing right tasks

Easier in games: Beat level 2, save, pause, defeat two droids, check high scores

Cognitive Walkthrough

You click the game icon on your PC and see a page with three buttons: play, settings, exit… State diagrams Flowcharts Control schematics

Figure out UI functionality Plan overall flow of experience

Physical Walkthrough

Paper prototype Loose menu/UI layout What fits on a page? What’s not allowed? (Error messages?)

Mind what requires explanation Not at stand alone point, but insightful Can warn about future problem areas

Heuristic Evaluation Many problems are universal and fall into

consistent categories Approach 1

• Look at categories, seek problems Approach 2

• Find problems, organize with categories

Hone in on problem areas Focus on most severe issues

Frequency, impact, persistence, aesthetic

Heuristic Evaluation Cont’d Simple and natural dialog Speak the user’s language Reduce user memory load Be consistent Provide feedback User control and freedom Flexibility and efficiency of use Help users recognize, diagnose, and recover from errors Prevent errors Help and documentation

Jakob Nielsen’s 10 Usability Heuristicshttp://www.useit.com/papers/heuristic/heuristic_list.html

Notes on HE Can be done by creators or observers Each person should go through twice

1 - figure out the general layout, understand the overall concept (“get” the mechanic)

2 - go through again checking for the heuristics For each problem, indicate location, type,

severity, and possible fixes Use this method to organize your To Do. Finds many errors, types of errors, good!

Nielsen’s Guidelines Learnability: How quickly can users understand

your main mechanics? Efficiency: How effectively can users perform the

tasks they want to do? (Make sure the challenge is in the right place)

Memorability: How easy is it to pick up the game again after an hour, day, month off?

Errors: How often do users make errors (is death an error?), how severe, and how easily can they recover?

Satisfaction: Do people like your game?

A note on evaluation

Most of HCI is driven by EFFICIENCY Games are not classic HCI targets

Emotion, enchantment, flow Experience vs. product “Intentional Inefficiencies”

Watch reactions as much as actions. Face vs. Screen

Observation Screen

Unexpected behavior Glitches/bugs “Think aloud” > “Why did you do that?”

Face Frustration Focus Fiero! Enchantment

Surveys

Why?! “Cool game” “Yeah it was great” “I really liked it”

Focus attention Increase feedback Open-ended questions

Forums

Games benefit from dev/user discussion Unity3D Free iPhone Playhaven Get friends involved in the iterations Find/fix bugs as they appear

Summary

Different methods, different information Tasks vs. Freeplay Walkthroughs, cognitive and physical Heuristics and design sensibilities Efficiency vs. Experience Observation, Surveys, Forums

Example Game: Influence

Half the Class on Tasks Half the Class on Freeplay Who finds the most heuristics?

Influence - Tasks Pick an allegiance Max the # boids in a game Play as the violin Have 6 adversaries Play a multiplayer game with at least

three other players Record the maximum amount of time you

survived a multiplayer game Play a game in respawn mode