What You Don't Know Is Hurting...
Transcript of What You Don't Know Is Hurting...
![Page 1: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/1.jpg)
What You Don't Know Is Hurting You: How Aggressive User Research
Improved Resistance 3
Drew Murray Creative Director, Insomniac Games
Lead Designer on Resistance 3
![Page 2: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/2.jpg)
![Page 3: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/3.jpg)
![Page 4: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/4.jpg)
![Page 5: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/5.jpg)
![Page 6: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/6.jpg)
External Playtest
External Usability Test
Developer Playtest
Developer Usability Test
![Page 7: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/7.jpg)
User Research Methods
Key questions:
1. Who are you testing: Real gamers or developers?
2. Aspect of experience: Affect, Behavior or Cognition?
(feeling, doing, or thinking?)
3. Category of data: Qualitative or quantitative?
4. How are you getting data: Observation, metrics,
self-reporting in situ, self-reporting post hoc?
![Page 8: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/8.jpg)
Usability Testing
![Page 9: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/9.jpg)
Usability Testing
RITE Method:
Rapid
Iteration
Testing
Evaluation
![Page 10: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/10.jpg)
Internal Usability Tests
![Page 11: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/11.jpg)
External Playtesting
![Page 12: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/12.jpg)
Internal Company Playtesting
![Page 13: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/13.jpg)
All Methods
![Page 14: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/14.jpg)
Usability Testing
+ Momement-to-Moment Player Experience
+ Cheap and (Relatively) Easy
+ Flexible - Not Focus on Emotional Response - Limited Sample Size
![Page 15: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/15.jpg)
Resistance 3 Usability Research Goals
![Page 16: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/16.jpg)
Core Shooting Controls
![Page 17: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/17.jpg)
Criteria for Good Aiming Controls
Balanced Unbalanced
Tight
Loose
Balanced + Tight + Fast Correction
Fast
Correction Slow
Correction
![Page 18: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/18.jpg)
Testing Approach
![Page 19: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/19.jpg)
![Page 20: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/20.jpg)
![Page 21: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/21.jpg)
![Page 22: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/22.jpg)
![Page 23: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/23.jpg)
External Playtest
External Usability Test
Developer Playtest
Developer Usability Test
![Page 24: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/24.jpg)
“Right off the bat, Resistance 3 controls wonderfully. . . .
The shooting mechanics are tight and responsive. . . .”
- IGN
![Page 25: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/25.jpg)
Usability Testing Tips
Work with an Expert
![Page 26: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/26.jpg)
Usability Testing Tips
Force the Player to Focus on What You’re Testing
![Page 27: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/27.jpg)
Usability Testing Tips
Build the Foundation Before the Penthouse
![Page 28: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/28.jpg)
Usability Testing Tips
New Employees = Gold
![Page 29: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/29.jpg)
Group Playtesting
+ Metrics
- Amount of Time Involved
![Page 30: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/30.jpg)
External Playtest
External Usability Test
Developer Playtest
Developer Usability Test
![Page 31: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/31.jpg)
Appropriate Research for Group Playtesting
Individual Feature Comparison
Pacing and Difficulty Tuning
Scores Generally Moving Up?
![Page 32: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/32.jpg)
Benefits to Developer Testing
Testing Prototypes and Speculative Gameplay
Asking Detail-Oriented Questions
![Page 33: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/33.jpg)
Individual Feature Comparisons
![Page 34: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/34.jpg)
Individual Feature Comparisons
April 2011
Survey Results
![Page 35: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/35.jpg)
Individual Feature Comparisons
April 2011
Survey Results
![Page 36: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/36.jpg)
Individual Feature Comparisons
April & May 2011
Survey Results
![Page 37: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/37.jpg)
Individual Feature Comparisons
April & May 2011
Survey Results
![Page 38: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/38.jpg)
Individual Feature Comparisons
Atomizer
![Page 39: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/39.jpg)
Individual Feature Comparisons
Cryogun
![Page 40: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/40.jpg)
Individual Feature Comparisons
Sledgehammer
![Page 41: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/41.jpg)
Challenge / Difficulty Tuning
April 2011
![Page 42: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/42.jpg)
Challenge / Difficulty Tuning
April 2011
May 2011
![Page 43: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/43.jpg)
“. . . Resistance 3 possesses one of the most ingenious
collections of weapons ever assembled in a video game; a
veritable wheel of abject destruction which inspires a degree of
familiarity which very nearly resembles parenthood.” - Joystiq
![Page 44: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/44.jpg)
Group Playtesting Tips
Plan Ahead
![Page 45: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/45.jpg)
Group Playtesting Tips
Identify Features with Pictures for Surveys
![Page 46: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/46.jpg)
Group Playtesting Tips
Tailor Questions to Audience and Testing Method
![Page 47: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/47.jpg)
Group Playtesting Tips
Make It Easy for Players to Take Your Survey
![Page 48: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/48.jpg)
Group Playtesting Tips
Automate Everything (But Manually Check Results)
![Page 49: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/49.jpg)
Did It Work?
“Best Resistance yet. . . by far.”
- Playstation LifeStyle
“It’s easy to say that this is the best Resistance
title.”
- 1UP
“This third title is easily the best yet.”
- Play.tm
![Page 50: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/50.jpg)
Was It Worth It?
![Page 51: What You Don't Know Is Hurting Youd3cw3dd2w32x2b.cloudfront.net/.../2012/03/Murray_Drew_WhatYou… · - Playstation LifeStyle “It’s easy to say that this is the best Resistance](https://reader033.fdocuments.in/reader033/viewer/2022051914/60051dc729930750b869c211/html5/thumbnails/51.jpg)
What You Don't Know Is Hurting You: How Aggressive User Research Improved Resistance 3
Drew Murray [email protected]
Additional Information on Game
User Research: Bill Fulton
www.RoninUX.com