Copyright T. Komischke, H. Strub, T. Sachs, J. Chin 2014
Lect 13: Theory & Practice of Usability Studies
User Experience Design IIIJohn Chin, [email protected], 2014
Contact Information
Copyright T. Komischke, H. Strub, T. Sachs, 2014
An broad overview of context- Who, Anyone- When, Anytime- Where, Anywhere- Why, Motivations- What, TaskThe missing link - How Along with some depth on many topics
2
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Doctor Who? Who is John Chin?Education
Ph.D. in Cognitive Psychology specializing in Human Factors Human-Computer Interaction
University of Maryland, College Park (advisor: Kent Norman)
Work Experience
Currently, User Experience Strategist @ Verizon Wireless User research, Writing UI requirements, Usability testing, Accessibility, Connected Car, Messaging, 3 patents pending
HiTech – Contractor Homeland Security, TSA Cognetics – Trizetto Group Avaya, Alcatel-Lucent, AT&T Bell Labs, Telcordia Lewis and Clark College
3
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Enough about me, what about you?Who are you?
Education and Work experience Interests and specialty
Why are you here?
Goals and Objectives Career development, job?
How can I help?
Needs and Aspirations Networking
4
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Bottom line goals… 1) Be neutral and stick to the facts 2) Advocate for your users, all of them
Recruit representative sample of eventual product/system users Participants – users testing the design/implementation Audience – observers (possible development team members) Report findings and possible solutions to readers or stakeholders
Gain better insights and understanding of the issues Hope to find possible solutions Few hard and fast answers: mainly tradeoffs
5
Planning a Test
Who: Low vision or Blind user Where: Standing on the sidewalk at a bus stop When: During rush hour in the morning What: Pedestrian getting directions to library Why: Pick up model created on 3D printer at library How: Mobile device navigation application
6
What are the research questions
Research questions are not tasks Key questions drive focus of the testing and tasks
Research: Can people successfully order products? Will customers be able to pay their credit card bill? Will radiologists be able to diagnose cases, without errors,
efficiently? Will consumers like this better than their current camcorder?
Tasks: Typically tasks should be frequent and/or important How long will it take customers to find the due date for their
current statement?
7
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Where is project in the product life cycle? Early?
Are there requirements that need to be gathered? Or are requirements finalized?
What’s the shape of design concepts? Is this the next version or a new product? Some will say “too early for usability” since much is undefined
Some will say “Do a (user-centered) focus group instead” Not yet 100% clear what it does Bottom line: A lot is vague, testing can have big impact,
but make sure you’re exploring relevant issues
Middle of process? Requirements finalized Screens rough, or designed for usability Lots already determined: Use it. [or not, at your own risk] Bottom line: Testing can have moderate impact, Fast
turnaround important. Choose issues where you’ll be listened to.
8
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Where is the project in the product life cycle? (continued…) Late?
UI is likely designed Possibly even coded
No leeway on requirements (functionality) Delivery possibly even scheduled What is the goal?
If the team will update the UI, could have a meaningful impact Fitting some request “It needs usability” No-win if in “police” role Stakeholder may be looking for “lipstick on the pig”, and endorsement
that “Usability says it’s okay” If it misses the mark, you’re in a no-win situation Big risk: once it’s delivered, product manager might not want to change
UI once existing customers invested time to learn it. Bottom line: be diplomatic & realistic. 1) Consider risks to
company if don’t fix enough, 2) what can be fixed later, and 3) process so won’t repeat this mess next release or product.
9
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Where is the project in the product life cycle? I prefer early-middle, or early Beware being brought in late: a no-win game
Especially if project planned without user research. No resources to create prototype (and no time) No resources to incorporate results (or time) No one might care Even if you have a sponsor, you’re being set up for a battle Focus on your data. It’s a business decision re what to do.
Be careful – think high level I’ve been surprised many times – people and organizations that
seem to have mature processes “organizations” – since companies have silos: one or many
organizations in a company may be awesome. Be careful every time you meet a new one.
10
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What do you know? 1) The human (in general) Topics include:
Cognitive processor (and memory) Perceptual systems Social interaction ( Emotion Other topics: Persuasion, attraction, etc.
If you have time to read, get applied books on these topics For topics you really like, get theoretical books; read
research papers. If you don’t already have a good book list, I’ll have
suggestions next week
11
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What do you know? 2) Your business:
If a consultancy (or an experienced department) How does your firm approach problems? Why do customers choose it? Know previous reports – especially with your client Know how to leverage templates: research plan, test documents,
reports Practically: consider ways to promote future work for colleagues – both
your team and other teams [without compromising my integrity]
12
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What do you know? 2) Your business
If you’re an employee: Get to know others in complementary departments
Market Research (make friends, which also reduces risk of being seen as competition)
Customer Service (Call Center “top 10” lists) Meet, go to lunch, read reports
VOC (Voice of the Customer) data Becoming increasingly popular Whatever you have available.
ForeSee:Survey at end of visits
13
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What do you know? 2) Your business, if you’re an employee, continued
VOC data, continued OpinionLab
Input possible on any page Works best with meaningful URLs
Beware: Length. Tempting to continue asking questions, but not paying
customers Order matters: What goes early late, which questions go before others
14
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What do you know? 2) Your business, if you’re an employee, continued
Analytics (i.e., big data)
15
Copyright M. Tremaine, T. Sachs, A. Melewski 2012
This is a sample, not an Adobe endorsement
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What do you know? 2) Your business, if you’re an employee, continued
Analytics (i.e., big data) The ability to track… Beware: What the data makes easy, versus what is of interest
I have been able to track interactions on a page (sometimes every page), but cannot track a user through paths
Who are your company’s customers? Who use online systems? How do people access your stuff?
What are trends? Social networking analysis: As many as company follows Reports: JD Power, Forrester, Keynote, etc. Data collection services, and reports, (usually not cheap) Technology trends
The continuing rise of mobile, tablets The impact of OS, browsers, etc. Zeitgeist: e.g., Responsive Web Design
16
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What will you test with? (1 of 3) Paper prototype?
Cheap, usually easy to create, conceptual Who will create? Will it be respected internally? “Photoshop” [i.e., online] paper prototype more “real” than paper Paper does communicate “conceptual”, room for change
I usually test paper as concepts that extend prototype(s)
17
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What will you test with? (2 of 3) Visio? (or similar, PowerPoint can be fast)
I’m not a “designer” but I can design fairly well in Visio If my concepts, I’m in control: can test what I want (or edit myself) If my concepts, distracting from focusing on testing
Key point: Hyperlinks, so it looks like it’s functioning More about screens than functionality (won’t function well, i.e.,
fields cannot be filled in)
Something that really runs?: Html, iRise, Axure, Dreamweaver, Ruby, etc.
Functionality is great addition to realism, believability
18
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What will you test with? (3 of 3) Possibly bad addition to cost, time, and pain, since
someone has to build & support the prototype. Arranging for someone to create the prototype can be hard, takes
time. Don’t assume getting resources will be fast/easy.
For many audiences, many products/systems, this is a necessity
Even more important for mobile UIs – I believe form factor really matters for mobile What do you think about where mobile is tested? [lab vs. field]
Trend of hybrid device approach: Big addition to complexity Prototype web, tablet, phone? iOS ? (pre-7 & 7?), Android (“flavor”?), Windows mobile,
Blackberry? App versus mobile web?
19
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How will you record the test? Screen critical – see what happened Audio critical – hear what happened,
High quality audio often undervalued. Stereo audio recordings can be especially useful, since with
headphones you can pick out voices
Very useful: The participant’s face, to see emotions Still useful – hands on keyboard, on artifacts, etc.
Mobile technology especially challenging to record Harder to get good screen image Want to see what hands do, but hands/fingers cover the screen Still want to see face of the participant
20
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How many participants to run? (1 of 3) Nielsen’s rules of thumb:
5 or a few more. Go higher if test has truly different groups
Consider more rounds of testing, if prototyping and time are available I often have trouble getting prototyping for multiple
rounds of testing) Bump up numbers a little if only one round of testing
21
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How many participants to run? (3 of 3) Hank’s rules of thumb
Arranging a usability test is hard Unless really simple, run at least 3 days
60 minute sessions: 15 participants 90 minute sessions: 12 participants
Beware going > 4 days at a crack – exhausting, and too much to analyze Too much for industry people to pay attention to
If need more people, run another series Or outsource the usability
22
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How many participants to run? (3 of 3) How many customer “demographic groups” to recruit
Rule of thumb: shoot for at least 3 people per demographic
If differing opinions, still have a tie-breaker If a no-show, still have 2 If it’s a key customer group (i.e., “prospects”) and site is
for them, increase or subdivide that group No point in testing 12 groups for study with 15 participants
23
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How much time per session? Guesstimate: How big/complex does the UI feel? If testing in cafeteria (or some place public) keep sessions
really short To cover more ground, lots of sessions covering different tasks
In lab, I believe no shorter than an hour Lots of overhead, effort, to recruit, to prep, to get observers, to
analyze
I find it hard to go > 1-1/2 hours Sessions are tiring, for participant, moderator, observers Hard for many types of participants to get lots of time during
business hours – might get more strange people > 90 minutes, schedule in a break (which lowers efficiency)
I’ve seen 2 and 2-1/2 hour sessions (and no break) in last year – and they worked, but moderator knew domain well
24
Copyright T. Komischke, H. Strub, T. Sachs, 2014
When to schedule sessions? Easiest: during business hours
If live observers, much better to have straight days & business hours
But will you get enough of “right” people?
Who are target participants? What are their days like? Many jobs: work day Physicians: One early (before 9) one at lunch, 1-2 starting at 5 pm
Think about yourself. I rarely schedule before 10 in Manhattan since hard to get there and lots to get lab ready.
If won’t have live observers, can run any time, including a few per day at odd hours
For some groups, consider weekends But if you have family or other personal issues (like you have a
life), get more money to pay participants
25
Copyright T. Komischke, H. Strub, T. Sachs, 2014
When not to schedule sessions Consider observers as well as participants Holiday season is hard If winter, have backup plan If big storm might arrive during test, have backup plans Allow time for analysis, and for results to be incorporated
Often have designers responding before discussion.
26
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How many sessions to run per day? Don’t make testing days too hard on anyone (including the
moderator) Want to be as alert, as good of a moderator for first and last
session Breaks between every session gives flexibility for buggy
prototypes, late participants, rebooting computer, trying something new, etc 60 minute sessions: up to 5 per day
Breaks of at least 30 minutes per session (even if just to allow for lateness), 60-90 minutes for lunch
90 minute sessions: up to 4 per day These are more tiring, make sure you have break time
Schedule lightly on first day of testing, to allow updating of everything – even your discussion guide, and to plan around unexpecteds When possible, run 1 or 2 participants first day, after lunch
27
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How to recruit participants? (1 of 2) What are you testing? How specialized the community?
What’s your budget? How much time do you have? Many options, all involve tradeoffs
Professional recruiters: For many types of consumers, and some target audiences (like MDs) Most say they’ll recruit anybody for any reason Realistically, they have specialties Most have “lists” of known people, who may be semi-professional test
takers (some seem jaded) Recruiters aren’t cheap
Lists of customers, you contact yourself If you want to test improvements to Emerson’s HMI, want people who
know today’s Emerson HMI Beware: lists can take a LONG time to get (6 weeks or more)
Referrals from sales staff, or from technicians who know their “cool” and “boring” customers
Subscriber lists, Professional Society members (especially valuable if want special expertise)
28
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How to recruit participants? (2 of 2) Recruiting from your web site
But any issue having “customer information”?
Waiting room at medical clinic Your card is given out by someone trustworthy (like a
doctor) “Mechanical Turk” at Amazon Craig’s List, or similar ads Your friends/acquaintances probably not diverse enough.
Beware effects of participants feeling obligated
Children can be especially challenging You want parents to be nearby so everyone is trusting, but you
have to keep them occupied
29
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Who to recruit If clear that there’s one demographic (or “persona”), go for
it But consider going for sub-demographics as more participants
I usually shoot for up to 3, maybe 4 max, demographic groups. Product space may be much broader Target groups that are truly different, in parameters that matter
to your UI Many factors can be targeted independently, with same
participants. E.g.: Ranges for age, education, income Separate from experience in what you’re testing
Avoid trap of recruiting only the young/well-educated They look nice on video, have great things to say, but… Unless all users will be young and well-educated I learn a lot from diversity
30
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Who to recruit: Common screening criteria (1 of 2) No one in your industry, or with a household member in
your industry No one in design or usability Always ask for a range, within whatever criteria.
If open to unemployed or retired, they’re probably easier to find for “business hours” studies than people who work, so “up to 2”
Internet skill: Want people who use web themselves, often, familiar
Age: cover diversity of your user community I’m pushing for older but not above 69 Many older users, but tend to talk more, and therefore get less
done (in my experience) I’ve had most dishonesty with elders: age (not be too old), internet skills
Consider gender distribution
31
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Who to recruit: Common screening criteria (2 of 2) Education: I’ve known researchers who screen for minimum
of some college for general consumer products. Getting some without college is more honest test of usability, for
those with low literacy skills – who buy and use things I’m often fascinated with responses of those with least education
Ethnic diversity Income: As low as appropriate. If you want some truly
wealthy people, might need to specially recruit and pay more
But can ask for people whose speech is easy to understand (critical)
Where they live: City vs. suburb; and which city (Philly vs. NY)
32
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Special participants: Children, Disabled, Employees Children:
Want a witness, at all times Parent should be nearby Make it fun Recruit so you’ll get interested kids
Disabled Hard enough to get disabled with full cognitive skill Check literature if recruiting disabled without full cognitive skill
Employees Is it their choice? Coerced? Impact of losing time on job Fear if criticize system, or people, when it’s recorded
33
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Possible Forms for participants (1 of 2) Informed consent form has two goals:
Inform participants of what they’ll be doing, realistically, any risks (even of embarrassment), and benefits (often, interesting tasks; chance to help change a product; and get paid)
Consent: So it’s clear what they’re consenting to
Media Release: That you are recording them How recordings will be used (for study, nothing else) Can protect you as well as participants: if colleague loves a clip-
wants to show at a conference (“Sorry!”) If session is that good, can go back and pay more, or recreate the
scene with actors
34
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Possible Forms for participants (2 of 2) NDA (Non-disclosure Agreement)
If you’re showing proprietary concepts Explain NDA too – so understand I’ve heard that a properly written and executed NDA protects
even if people talk – you did your due diligence
Liability release (?) In case incidentally hurt when coming to your facility
Include conditions that lawyers want But fight for plain language Long forms are harder to understand –consider more short
forms than fewer long forms
35
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Don’t fight the law (but then I’m married to a lawyer) This is probably all new for most corporate council Draft pretty good sample forms before approaching
Shows you know something Easier to edit than start from scratch
Be able to explain what’s there, why, who and what your intentions are
Provide articles on what you’re after, or books Negotiate, but with the company’s best interests in mind
36
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How much to pay participants? Goldilocks rule of thumb:
Not too much, not to little
If doing internal testing on employees, might be limited by corporate rules (participants already missing work during work time
For external participants, ask your recruiter for advice Consider adding in reasonable extra payment for
transportation/parking costs BTW, if paying a participant more than $500 in a year,
check with company HR about possible need to report “salary” to IRS Only an issue if have same person return
37
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Should I use floaters? Floaters are backup participants, usually recruited to cover
two sessions Extremely valuable in case of no-show Also, for when a participant is totally inappropriate But, who has time to hang out for 2 sessions? Be careful
I use best recruiter I can, request to go for 100% turnout I’ve had odd birds as floaters
38
Copyright T. Komischke, H. Strub, T. Sachs, 2014
How to record a test? Morae, by Techsmith: The “Honda Fit” of usability
recording: $1500 for a full license Records two streams, usually screen and webcam on face, as well
as an audio stream If built-in mic isn’t awesome, get a good external mic
1copy of Morae includes 1 copy of “Observer”, which lets people in another room watch live (delayed ~20 seconds)
Very reliable Not bad to learn, to set up Can take notes, and make marks, live (but I don’t see that
done) Will make highlights videos Pretty efficient storage, so many sessions can go on a DVD
39
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Recording a test – More on Morae Morae also good for portable lab – only need 2 laptops Requirements for computers not clear: doesn’t reflect multi-
core processors In my experience, Morae has run fine on machines well
below the “minimum recommended” capability
A hack to get 3 images is a hardware video mixer, that will superimpose for you.
40
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Recording a test: Lots of fancier software, but.. Pricier, need fancier computer and equipment, can be
testier I know many big companies use OvoStudios My personal experience: very nice & helpful, not as reliable
as I’d like But I’ve heard of companies that find it trustworthy Network to find something that works for your needs.
Consider: HD looks great, awesome detail Probably not needed for routine recordings of basic web sites
41
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Recording a test: Back to mobile How to get a good screen? Apple to HDMI to DVI cable, or
Apple TV hack if security allows License supports 3 images – I think all 3 are important [Note: target area – ask participants to keep the phone
THERE
42
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Other Technology Issues: Eye Tracking: compelling to watch, help observers stay
awake Beware of heat maps: If interrupt participants, etc. Magic is uninterrupted, enough people to get good trends, so
software can analyze what happened Web site also needs to support eye tracking: overlays, modals
mess things up Software only knows the URL, not what’s on screen
Emotion tracking: GSR (how emotional) and valence (+/-) Affectiva is a company to watch
43
Copyright T. Komischke, H. Strub, T. Sachs, 2014
The Moderator’s Guide (1 of 4) Introduction Why you’re there That this is usability so
A) Please speak aloud (we’ll practice), B) Testing the UI, not testing you, C) Don’t worry about errors since prototype (not real data or
system)
Ask questions (though I might not answer) Can stop at any time, for any reason
Pay at beginning of session: Show trust (people don’t walk away)
Plan the task list Based on the research questions Ideally, based on your knowledge:
What are the true top tasks (consider what Gartner, JD Power say)
44
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Moderator’s Guide (2 of 4) Think about time. Consider optional tasks to fill sessions for
fast participants. Or have an extra task or two, and prioritize list in pilot sessions
Moderator’s guide: By each task I like to have questions to probe, and suggested things to observe Participant’s guide just has the tasks – other stuff is deleted Edit moderator’s guide, either regenerate Participant’s guide, or
be careful on version control.
Questions to ask: Per task: Make sure to ask the goal of the task
Find a value? Perform a task Otherwise, think about what you’ll learn
Can ask questions like “How quickly do you expect to do this task? How easy do you expect this task to be?
After: How easy was it to perform this task? How often do you perform this task?
45
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Moderator’s Guide (3 of 4) Questions to ask: At end of Session
I like to ask: What did you like best about the prototype system? What did you like least about the prototype system? [Sometimes in form of “What were 3 best & 3 worst things?”]
The SUS is trendy [SUS = System Usability Scale] Original by John Brooke, 1986, DEC
46
Copyright T. Komischke, H. Strub, T. Sachs, 2014
47The original SUS
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Moderator’s Guide (4 of 4) Thoughts on SUS:
Alternates positive and negative questions, so respondent needs to be awake
Around since the 80’s, lots of experience in the industry A few common updates [i.e., “website” instead of “system”] Like school grading: 70 is okay, 80 good, 90 superb But maybe UI’s are better these days -> I’ve seen mainly upper
80’s or better
Net Promoter Score (NPS): How likely are you to recommend this product/company to a friend? Trendy, originally published in Harvard B-School Journal
Were participants supposed to learn something? Though be careful about testing learning in 1-hour long session,
unless that’s what system is for 1 hr not much time to learn, especially when many tasks
48
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Cautions on locations (1 of 2) Dedicated lab:
Will you use it enough? Perhaps try lending it out, but beware… Sufficient staff support? [reception & waiting area, space for
observers who have meetings, snacks & drinks and meals] Window: Great, but then keep observer room quiet and dark Need to keep equipment secure
Conference rooms and a portable Morae lab Still have to keep laptops secure Will you always get good rooms for test when you need them?
Others’ office Conference rooms: Will your Morae computers run on their
network? If not, how will they communicate? Fairly safe from interruptions, though.
Cubes: Beware interruptions… Phone will ring & cube’nik might need to pick up; Co-workers don’t recognize you as important
Offices: Probably okay as long as participant respects you
49
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Cautions on locations (2 of 2) Remote Testing
Success depends on installing web-sharing software, but software, computers and bandwidth are all pretty good Usually only 5 minutes, sometimes much worse
Better with web cam Surprisingly to me, rarely interrupted, but phone calls seem to be
respected
3rd party facility (i.e., market research company) Surprisingly pricey – be prepared for unexpected costs But they handle the details – REALLY nice When I use a facility, I have them recruit – so one point of failure
Even professionals sometimes use cafeteria, or Starbucks Best for quick “first impression” studies, lots of people in little
time Can recruit for more in-depth study (follow-up in lab…)
50
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Screen-sharing software, for remote live observers Easiest to use the corporate standard
It’s IT-endorsed No firewall issues Account likely easy to set up, and/or use If there’s a problem, you used what company wanted
Otherwise, I believe in WebEx & a telephone bridge Usually reasonably pain-free to install Performance (delay, quality, frame-rate) pretty good Not cheapest, but trustworthy is important to me
51
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Running a Test52
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Be like Johnny Carson(Used to host The Tonight Show) Always genial, friendly Show was about the guest(s), not him THEIR thoughts & opinions Johnny didn’t lead But he got guests to talk When there was a problem, he took the blame (never the
guest’s fault unless something terrible) Was incredible at improvising
For more on Johnny, watch for rebroadcast of PBS documentary
53
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Taking notes during: If you follow template strictly and have time, excel template
for all key questions and observation points is neat Time consuming to set up Not convenient to use when with participant
Can take notes with computer Not best for establishing rapport
Take notes at a constant pace (lots of them) So don’t communicate what you care about So writing when participant thinks they’ve said something
important
Compatriot’s notes (ideally, not in sight of participant) can be great, but everyone looks for different things Who will be lead on the report?
54
Copyright T. Komischke, H. Strub, T. Sachs, 2014
Things to watch during session55
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What to do if: (1 of 2) Participant is no-show
Ask to arrive 15 minutes early: time for forms, for bathroom, and to call recruiter before session time
Call recruiter – check on reminders Backup slots available?
If remote, can’t get to internet Common to get weird firewall issues Have a local copy
Prototype fails Computer fails Participant arrives quite late Running really behind. Participant’s a talker, trouble, started
late Participant wants bathroom break, but already behind
56
Copyright T. Komischke, H. Strub, T. Sachs, 2014
What to do if: (2 of 2) Early in session, can see that participant doesn’t fit the
study? It appears that participant lied to recruiter re something
key? (After all, these studies pay well and economy is bad) You give cash or check to participant at start of session, and
he/she gets up and leaves? Participant admits to having committed a crime on your
video recording? (“When I do my taxes, I don’t put in all the receipts”)
You have strong evidence that participant is victim of or commits abuse, or is considering suicide?
57
Copyright T. Komischke, H. Strub, T. Sachs, 2014
After a test Debriefs are great. Consider recording them if you don’t
take good notes. Write down first impressions of results while they’re fresh in
your mind. Get invoices in promptly: * participants *, recruiter, and any
vendor you might use again Return borrowed equipment promptly (before damaged,
lost) Listen to sessions as “background music”. You will hear
things Push self for rapid turn-around.
Consider rehash/highlights meeting immediately afterwards, recorded, to get impressions of observers
Tight deadline for report doesn’t allow forgetting But beware getting slammed if rest of your job fell behind due to time
away for testing
58
Copyright T. Komischke, H. Strub, T. Sachs, J. Chin 2014
Questions?