Remote Usability Testing

30
1 Remote Usability Testing December 5, 2009 Elizabeth Snowdon Usability Analyst, Sage Software [email protected]

description

It is important to learn to be flexible in your work and to learn tools that will empower usability analysts to be more effective in their roles. Also, in an age of diminishing corporate spending and rapid prototyping and testing, remote usability can save companies a lot of time and money.

Transcript of Remote Usability Testing

Page 1: Remote Usability Testing

11

Remote Usability Testing

December 5, 2009

Elizabeth SnowdonUsability Analyst, Sage Software

[email protected]

Page 2: Remote Usability Testing

22

About meSenior Usability Analyst at Sage Software

Previously worked in usability at high-tech company

Conducting usability studies since 2003

Page 3: Remote Usability Testing

3

Principles of Scientific Management

Page 4: Remote Usability Testing

4

Traditional Lab Testing

Page 5: Remote Usability Testing

55

• Remote usability• Advantages / Disadvantages• Case study• Tools

What I will talk about today

Page 6: Remote Usability Testing

6

What is Remote Testing?

• Testing sessions in which you and participant are not physically in the same place but are communicating via electronic technology

6

Page 7: Remote Usability Testing

7

Types of Remote Testing

7

Remote User Research

Synchronous

Asynchronous

•One on one and real time•Shared technology

•No moderator•Self-paced •Special tools record activities & feedback

Page 8: Remote Usability Testing

8

Current Usability Testing Trends

1. Movement away from lab testing

2. Increased interest in testing in users’ environments

3. Agile development methods

Page 9: Remote Usability Testing

9

Advantages• Lab not required

9

University of Leeds Human Perception and Performance:Usability Laboratory

Page 10: Remote Usability Testing

10

Advantages

• Expands sampling population

• Ease of recruitment

• Users in their own natural browsing environment

• Share their desktop with you

Page 11: Remote Usability Testing

11

Advantages

• Save money– Travel

– Equipment

– Recruiting costs

– Compensation

11

Page 12: Remote Usability Testing

12

Disadvantages

• Difficult to see participant’s screen

• Cannot see participants

• Problems downloading or installing desktop sharing

• Task scenarios disseminated in advance

• Participants interrupted or distracted

12

Page 13: Remote Usability Testing

13

When not to use

• When security is important

• When reliable high speed internet connection is unavailable

• For eye tracking studies

• When it is important to attend to user’s physical movement

13

Page 14: Remote Usability Testing

14

When to use

• Recruiting valuable individuals

• Minimal set-up

• For lone wolf consultants

• Rapid iterative development (Agile)

14

Page 15: Remote Usability Testing

15

Recruiting Users

15

Invitation + incentive

Screener ScheduleConfirmation +

Consent

Session Thank YouTasks

Page 16: Remote Usability Testing

16

Case Study

• Baseline study for Accpac

• Three personas – clerks, controllers, operations

• 24 participants – 20 remote

• US, South Africa, Australia, UK, Canada

• One moderator, one note taker, observers

Page 17: Remote Usability Testing

17

Results

• Accpac study completed in 4 weeks• Cost savings

– Travel– Lab set-up

• Time savings– Recruiting – Scheduling – Less overtime

• Bottom line - $50,000 saved

Page 18: Remote Usability Testing

18

Recruiting

• Internal recruitment, customer lists, Craigslist, email

• Interested parties must pass screener

Page 19: Remote Usability Testing

19

Other issues to sort out

• Obtaining informed consent and confidentiality agreement

• What computing and communications equipment do candidates have?

• Are they willing and able to download and run collaboration software?

Page 20: Remote Usability Testing

20

Typical Set-up

• Moderator / Note-taker – Screen sharing: WebEx– Recording: Morae – Speakerphone

• Participant– High speed internet access– Speakerphone or headset telephone

Page 21: Remote Usability Testing

21

Meeting room set-up

Page 22: Remote Usability Testing

22

What can go wrong?

• User forgets– Send reminder one day in advance

• No shows– Provide your cell # and get participant’s cell

and work #

• User cannot download plug in– Send participant WebEx compatibility test

check email in advance

Page 23: Remote Usability Testing

23

What can go wrong?

• User takes a long time to log onto WebEx– Add extra 10 minutes for WebEx set-up– Log onto session 15 minutes in advance– Get participant to log onto WebEx 10 minutes

in advance

• User brings along his co-worker– Explain 1to1 test– Reschedule secondary participant

Page 24: Remote Usability Testing

24

What can go wrong?

• Can’t see what the user is doing– Ask user to point with mouse

• Conversations– Mute your line

Page 25: Remote Usability Testing

25

WebEx Limitations

• Test! Test! Test! – Dry run

• Delays– Explain that there is a slight delay between

what they are seeing and what you see

• Cannot view hover overs– Verbally tell users what they are hovering

over

Page 26: Remote Usability Testing

26

Pre-test / Test checklist

Email scenarios Set-up system or web site Open up required documents Dial into conference bridge or participant Sign onto WebEx Facilitator script Start recording Hand WebEx control to participant

Page 27: Remote Usability Testing

27

Incentives

• Challenging for international users

• Email local Amazon gift certificates

Page 28: Remote Usability Testing

28

Resources

• IBM - Experience remote usability testing

www.ibm.com/developerworks/library/wa-rmusts1/• Remote Usability remoteusability.com/• User Experience magazine

ww.usabilityprofessionals.org/upa_publications/user_experience/past_issues/2008-3.html

• Moderating Usability Tests: Principles and Practices for Interacting by Loring and Dumas

• Usability Engineering by Jakob Neilsen

Page 29: Remote Usability Testing

29

Remote tools

• Liz Bacon listing http://remoteusability.com/remote-testing-tools-round-up-by-liz-bacon/

• Bolt Peters http://remoteusability.com/tools/

Page 30: Remote Usability Testing

30

Questions

• Feel free to connect with me at [email protected] or

elizabethsnowdon.wordpress.com/