Usability Testing with People with Disabilities

Creative Commons – Some Rights Reserved

1

Usability Testing with People with Disabilities

Image: xxxxx

Kate WalserCX [email protected] • @kwalser

Slides: http://www.slideshare.net/kwalser


2

2

» 18 years in usability, accessibility and user experience design

» Member of TEITAC, or Section 508 / Section 255 refresh committee

» Principal consultant at CX Insights, the user experience division of Tritus Technologies, Inc.

» Originally from upstate NY, now call Fairfax, VA (near DC) home

Kate Walser

[email protected]


3

SCHEDULE

Learning & PrepTime Duration Topic

2:15 – 2:25 10 minutes Introductions, Objectives

2:25 – 2:35 10 minutes Accessible & usable

2:35 – 2:50 15 minutes Planning & logistics

2:50 – 3:00 10 minutes GROUP EXERCISEPlan your test

3:00 – 3:10 10 minutes Moderating & observing

3:10 – 3:20 10 minutes GROUP EXERCISEPractice

3:20 – 3:30 10 minutes Break

3:30 – 3:40 10 minutes Welcome participants


4

SCHEDULE

Applying What You LearnTime Duration Topic

3:40 – 3:55 15 minutes Session #1

↳ 3:55 – 4:05 10 minutes Debrief, switch


↳4:20 – 4:25 5 minutes Debrief, switch

4:25– 4:40 15 minutes Session #3





5:20 – 5:30 10 minutes Wrap-up, Questions, Final thoughts


5

Accessibility in a Nutshell


6

6

ACCESSIBLE

3 Key Principles

It’s accessible if I can…1. Get there2. Know where I am3. Know what I can

do- Jim Thatcher


7

DEMO

Using a screen reader


8

NavigationMegamenus, Repetitive menus, Hover / Mouse-only access, Clear focus, Tap targets

https://www.whitehouse.gov/


9

FormsCues, Formats, Feedback


10

Dynamic AreasAutocompletes, Data-dependent forms, Pop-ups, Surveys, “How can I help” chat


11

Multimedia & Visual ElementsMaps, Charts, Audio-video


12

VIDEO

Blind film critic

Play videoSource: https://youtu.be/c0nvdiRdehw

https://youtu.be/c0nvdiRdehw


13

GOAL

Accessible and usable

Photo Credit: Actions vs. Intentions: 20111219_326, by Gerald Fittipaldi, https://flic.kr/p/kvnGo2


14

Usability Testing SUMMED UP


15

Why we test

» Observe– Representative users trying site (vs. just asking)– How easily they meet their goals– What causes confusion or problems

» Inform design– Confirm or challenge design assumptions– Find and fix issues before launching

» Make an impression– On us – Remember what participants did and said– On them – Act as a representative of organization, product


16

Expected Use vs. Actual Use


17

WHAT WE LEARN

Behavior & Actions

» “Oh by the way…”’s» Environmental factors» Shortcuts, preferences» Power user habits


18

USABILITY TESTING WITH PEOPLE WITH DISABILITIES

Make it accessible first

» Usability testing ≠ accessibility audit» Usability testing requires…

– All elements reachable– Context, feedback available– Site / app can be used with assistive

technology


19

When to Conduct Usability Testing

Apply UX, design

principles

Fix a11y problems

Conduct usability testing

Conduct testing

with PWDs


20


What’s the user experience like?

» How will a person with disabilities use the product?– Assistive technology use and settings– Other resources they use

» How usable is the product by people with disabilities?– Content, forms without surrounding visual context, proximity– Navigation– Truly accessible vs. just meets the standards

» How engaging and effective is your content strategy?– Images only vs. engaging context + images– Search engine readiness (headings, link labels)– Great content flow


21


What We Can Learn

» Different methods for finding information» Whether content, navigation are usable

(not just accessible)» Adequate context for completing tasks


22

Planning & Logistics2:35 – 2:50


23

Choosing Participants

Example Component Senses Include users who are…Website, Mobile

- Visual interface- Links, menus- Forms

- Vision- Way to navigate

(mobility, speech)

- Blind- Vision impaired- Mobility impaired

Multimedia - Visual interface- Sound- Way to select

options

- Vision- Hearing- Way to navigate

(mobility, speech)

- Blind- Vision impaired- Deaf- Hearing impaired- Speech challenges

Phone - Speech and audio feedback

- Hearing- Speech (or way to

indicate option)

- Deaf- Hearing impaired- Speech challenges


24

24

Location OptionsIn-person» Pros

– Meet user & interact directly– Can see body language– Observe user’s set-up more

easily» Cons

– Takes more time for participant (travel + session), recruiting

– Requires accessible space– Need more time to arrange,

complete– Include local participants only

Remote, online» Pros

– Occurs at user’s convenience

– Makes recruiting easier– Invite users from further– Involves fewer logistics– Schedule, complete quickly

» Cons– Requires accessible tools– Miss body language– May not see person’s face


25

IN-PERSON TESTING

Environment & Setup

» Location becomes even more critical– Is it reachable by public transportation?– Is the building itself accessible?– Are the signs to find the room accessible?

» Room setup matters– Is there space to navigate with wheelchairs or walking devices?

Is there space for a service animal to lie down near its human?– Are tables or desks wheelchair accessible?– If visually impaired, can the person sit close enough to the

monitor to view the screen? Can you mirror the screen on another monitor for the moderator to view?

– How close are other workstations if screen readers will be used?


26

IN-PERSON TESTING

Technology

» Will participant bring their device?– Lets them use and you see their preferred settings– May wish to bring earphones

» Will participant need device / AT supplied?– Ask about OS, browser, AT preferences during recruiting– Leave some time for set-up once participant arrives


27

27

REMOTE TESTING

Technology

Moderated» Set up a web conference

– Use an accessible web conferencing tool – GoToMeeting, WebEx, Skype (Premium for screen share + video)

– Set up phone conference or use included audio

» Leave time for participant to download plugins

» Ask about video

Unmoderated» Use an accessible

testing tool– E.g., Loop11


28

Recruiting» Allow more time to find participants

– Screener + disabilities

» Look for community groups and ask– Meetup, Facebook, Twitter

» Use specialized recruiting services– Knowbility AccessWorks portal

» Work with outreach groups– Advocacy groups– Employment non-profit groups– Colleges and universities


29

Scheduling

» Allot longer session times– AT / device set-up, adjustments– Listening with screen reader

» Schedule breaks between sessions– AT can be overwhelming for users

who don’t typically use it


30

SCHEDULING > CONFIRMING

Give participants a familiar voice

» If possible, confirm the session yourself» Whether phone or email, introduce

yourself– “Hi, this is Suzy Thompson from the Acme usability testing. I’ll be your moderator and wanted to give you more info about the session and see if you have any questions…”


31

SCHEDULING > CONFIRMING

Details

» Explain what will happen and ask about specifics– “We usually provide handouts for users to read that describe

the scenario. What’s the best way to provide that to you?”» Describe the setup, and include accessibility

specifics in your confirmation emails– “We’re on the 3rd floor. There’s a ramp to the right of the front

doors. The doors automatically open. There’s an elevator bay in the lobby to get to our floor.”

» Ask if they will bring anything– Service animal (would they like a rest or water stop for the

animal?)» If food will be served, ask about dietary

restrictions


32

Additional logistical considerations

» Is an area available to walk service animals?

» Will you have pre- / post-session surveys?


33

GROUP EXERCISE #1 – 2:50 – 3:00

Create your test plan1. Find partners

2. Pick your test site, application

3. Choose 1-2 tasks you want to test


34

Moderating & Observing3:00 – 3:10


35

35

Make participants feel valued and important

» Welcome and guide participants

» Memorize the script» Learn negotiation tactics

– Maintain an open mind– Manage challenges that arise

(Handle surprises without disruption)Photo credit: “No Cavities!” daveparker

http://www.flickr.com/photos/daveparker/


36

36

Moderating Change #1

Anticipate AT’s impact on the session

» Practice listening to JAWS or other AT while someone is talking

» Know when the AT will affect how you interact with the participant

» Expect user to listen first» Anticipate user AT

shortcuts» Be patient and don’t expect

to understand everything


37

Moderating Change #2

Provide accessible materials electronically

» If user will bring own device…– Option A: Email tasks, survey links ahead of time– Option B: Post the tasks, survey and give user an easy link to

access them

» If you supply the device…– Place text file with the tasks on desktop– Add browser shortcut to survey

» Use alternate versions to randomize task order– Each participant may have different task order in their file


38

Moderating

Otherwise, little changes

» Act like you normally would– Treat them like you would any other colleague or friend– Ask first before assuming they need help– “See” and “look” are still ok

» Ask user to tell you what the task is after reading it

» Remind them not to explore (unless that’s the task)


39

39

Observing

Watch for usability problems

» General issues» Issues made worse by

poor accessibility or interaction with AT– Missing information,

problems navigating– Look for issues caused by

the presentation» “Surprises”


40

Observing

Tools help

» Add web conferencing option– Participant accepts invite, shares screen– Start sessions on other machines

» Use AirServer (or other tools) for mobile sessions to screencast to other device


41

41

» Give them time– Time to listen to AT– Time to respond

» Clear browser cache between users!» Remove outline:0

Observing

Additional tips


42

GROUP EXERCISE #2 – 3:10 – 3:20

Practice moderating1. Welcome and thank the participant for coming2. Describe the task as briefly as possible3. Ask them about their AT / set-up4. Run thru the site or app you picked

1. Can you reach everything? Can you reach things if you zoom in?

2. Is the focus indicator outlined?3. Are there any dynamic elements that will be trouble for

the participant?


43

Windows / Screen Reader Tips

1. Tab / Shift+Tab to tab through links, buttons2. Enter key to activate3. Spacebar to select checkboxes4. For *accessible* megamenus, try left / right

arrow keys

More infohttp://www.microsoft.com/enable/products/windows7/


44

Apple / VoiceOver Tips1. Command + F5 to activate2. Ctrl to stop speech3. Tab / Shift+Tab to tab through links, buttons4. Enter key to activate5. Spacebar to select checkboxes6. For *accessible* megamenus, try left / right arrow keys7. Ctrl+Option+F2 to hear window summary8. Ctrl+Option+Command+H to move to next heading9. Ctrl+Option+Command+J to move to next control

More infohttp://help.apple.com/voiceover/vo/en/VOKeysColor_1.htmlhttp://www.applevis.com/guides/voiceover-keyboard-shortcuts-mac-os-x


45

iPhone VoiceOver Tips

1. Set the Accessibility Shortcut to Voiceover2. Toggle on / off VoiceOver: Triple tap the Home

button3. Mute or unmute VoiceOver: Three-finger double-tap

More infohttp://www.apple.com/accessibility/ios/voiceover/


46

Android Talkback Tips

1. Press and hold the power button until you hear a sound or feel a vibration.

2. Release the power button.3. Touch and hold two fingers until you hear audio

confirmation (about 5 seconds).

More infohttps://support.google.com/accessibility/android/answer/6007100?hl=en


47

TEST PLANWhat do we want to test?

Website / app?Desktop, mobile, both?What senses are in play? (vision, hearing, speech)Which populations should we include?When do we want to test?What features, functions do we want to test?


48

TEST PLANWhere and how will we conduct testing?

In-person or remote?In-person

What location is accessible?Can it be reached via public transit?Is there a room with adequate space?Do we have AT available?

RemoteDo we have SW licenses?

How can we find participants?How can we advertise / promote the testing?


49

TEST PLANWhat logistical considerations do we need?

Who will schedule & confirm participants?If food will be served, ask about dietary needs.Who will greet them? (Ideally same person)Can we send any pre-test survey ahead of time?How / will we randomize the tasks?How will we provide the tasks to users? (e.g., Text file? link to list of tasks?)Will we use a survey tool for the post-test survey?If remote, have we made an easy link between the test and survey?


50

TEST PLANDo we need anything else for in-person moderating / observing?

Who will moderate the sessions?Do we need to record the sessions?Have we set up any web conferencing sessions so others can observe on their machines?Do we have new / clean earphones available?


51

Session 1 – 3:40 – 3:55

Product testedParticipantUser device, OSAT usedSpecial settings?Macros, bookmarks, style sheets?

Check whole page / context first?Use AT shortcuts?Links list, headings

If speech recognition, how do they navigate?Grid coordinates, Words


52

SESSION #1 DEBRIEF – 3:55 – 4:05

What did we learn?


53

Session 2 – 4:05 – 4:20





54


What did we learn?


55

Session 3 – 4:25 – 4:40





56


What did we learn?


57

Session 4 – 4:45 – 5:00





58


What did we learn?


59

Session 4 – 5:05 – 5:20





60


What did we learn?THANK YOU PARTICIPANTS!


61

Wrap-up1. Usability testing with people with disabilities is not the

same as an accessibility audit.2. When including people with disabilities, the main

adjustments involve recruiting and logistics.3. Understanding and prepping with AT will help sessions

go smoothly.


62

Thank you!

Kate [email protected]

Usability Testing with People with Disabilities

Technology

Transcript of Usability Testing with People with Disabilities