Notes are not enough! Why relying on your notes will lead you down the garden path.
-
Upload
ash-donaldson -
Category
Design
-
view
607 -
download
3
Transcript of Notes are not enough! Why relying on your notes will lead you down the garden path.
NOTES ARE NOT ENOUGH!
@ashdonaldson Design Research 2017, Sydney
Resurgence of research
I decided that we shouldn’t even look at the notes. We’d just go
with our gut.
Senior Researcher playing back findings
We can all capture stories…
…but are they true?
*sniff*
Communication feels intuitive…
…but it’s really complex
The great enemy of communication, we
find, is the illusion of it.
William H. Whyte
Communication can be critical
Melbourne tower, this is Alpha Echo Charlie inbound from the South at 5,000. Request descent to join downwind of zero niner at 1,000.
An interview is a directed conversation.J. & L. Lofland
How we think about thinking…
Mental models of the mind
Hydraulic
Mental models of the mind
Hydraulic Mechanistic
Mental models of the mind
Hydraulic Mechanistic Information Processor
Information Processor Model
- Fixed focus - Captures everything
Information Processor Model
- Fixed focus - Captures everything
- Logically interpreted - Systematically classified
Information Processor Model
- Fixed focus - Captures everything
- Logically interpreted - Systematically classified
- Stored faithfully - High definition
Information Processor Model
- Fixed focus - Captures everything
- Logically interpreted - Systematically classified
- Stored faithfully - High definition
How communication works…
It all starts with an intent
Sender
that intent exists within an environment
Sender
Interviewer’s environment
the intent needs to be expressed as an idea
Sender
Interviewer’s environment
Encodes
through a channel
Sender
Interviewer’s environment
EncodesChannel
carrying transmitted information
Sender
Interviewer’s environment
Encodes MessageChannel
which has to be perceived
Sender
Interviewer’s environment
Encodes MessageChannel Channel
[ attention is limited ]• We don’t have a fixed attention
• Serial switching: Can only attend to one thing at a time
• Easily distracted: Directed Attention Fatigue (DAF)
• Expensive and rapidly exhausted
• Affected by sleep, stress, nutrition, medications, age & exposure to external stimuli
[ perception is tiny ]• We don’t take everything in as input
• We perceive only tiny fragments
• Our minds construct a seamless movie
• We more readily perceive things we expect
• Pilots divide the sky into quadrants to scan for aircraft
• Look under a pier: you’ll notice more fish the longer you look
Inattentional Blindness: The Monkey Business Illusion - https://youtu.be/f94o3B3csYI
…and interpreted by the receiver
Sender
Interviewer’s environment
ReceiverEncodes Message Decodes
Participant’s environment
Channel Channel
[ cognition ]• Our brains do not function like CPUs
• Our cognition is based on pattern matching
• We draw heavily on past experiences and present cues
• We satisfice - stopping when we have enough information to make a conclusion
• We are subject to many predictable biases
Nothing is so difficult as not deceiving
oneself.
L. Wittgenstein
https://www.ted.com/talks/michael_shermer_on_believing_strange_things
[ memory ]• We do not store information like a hard drive
• Long-term memory is constructive
• The more you remember it, the further from the truth it is
• Short-term memory is fleeting
• Fills quickly (4 chunks - not 5 to 7)
• Gets replaced by the next thing we attend to
• Has limited duration (10 to 20 seconds)
Interviewer’s environment Participant’s environment
both parties have a history
Sender ReceiverEncodes Message DecodesChannel Channel
then a signal is sent back
Sender
Interviewer’s environment
Receiver
Receiver
Sender
Encodes Message
Message
Decodes
Participant’s environment
EncodesDecodes
Channel Channel
Channel Channel
Feedback Feedback
Noise Noise Noise Noise
…of course, there’s always some type of noise
Sender
Interviewer’s environment
Receiver
Receiver
Sender
Encodes Message
Message
Decodes
Participant’s environment
EncodesDecodes
Channel Channel
Channel Channel
Feedback Feedback
Noise Noise Noise Noise
Transactional model of communication
Sender
Interviewer’s environment
Receiver
Receiver
Sender
Encodes Message
Message
Decodes
Participant’s environment
EncodesDecodes
Channel Channel
Channel Channel
Feedback Feedback
Notes take effort
CPM-GOMS of a note
Add a third perspectiveInterviewer’s environment Participant’s environment
Notetaker’s environment
So what should you do?
1. Record your interview
When reviewing the footage I saw their expressions in the moments when I was looking
away. It was intense, and important to re-live some of these interviews seeing new
things.
Natalie Rowland Qualitative Specialist (redrollers)
Recording interviews: Tips• Craft a good recording release
• Aim for video. If not, use audio
• Skype or Facetime: We use eCamm’s Call Recorder
• Aim for a quiet, well-lit space in context
• Test your recording equipment
Recording interviews: Equipment
2. Get transcripts
We had 3 notetakers for one of our interviews (most
notetakers I’ve ever had) and I just combed the transcript. We missed about 15 critical
points that were said.
Alyce Lythall Senior Researcher (Tobias & Tobias)
Transcripts: Tips• Use participant coding e.g. T1P03
• REV.com US$1/min
• Clients appreciate research documentation
• Clean up your audio
3. Build in enough analysis time
Analysis: Tips• Create a cadence that allows digestion
• Allow at least 3x the research time
• Build the time into projects up front
• If you can’t sell analysis time, hide it
• Use Quicktime Pro or similar that will allow you to playback recordings at 3x
• Comb transcripts for insights and verbatims
Notes are not enough!• Recordings and transcripts give objective feedback:
• Re-live stumbles and correct next time
• Every interview recording is an opportunity to become a better researcher
• Recordings and transcripts = Good data in
Questions?@ashdonaldson
Tobias &
Tobias