Post on 04-Jan-2016
SEBASTIEN J ANINI Group Client Service Director TNS CONSUMER Sebastien has worked on the agency side for twelve years, with a career spanning various geographies, research practice areas and client sectors. At TNS UK, Sebastien is a Group Client Service Director for Consumer. His main role is to look at strategies for commercial growth, and to support the development of new structures to deliver commercial momentum. Sebastien has also direct responsibility for some of the largest UK Consumer accounts. Sebastien has been involved in promoting and supporting the development of Online research across the whole UK business, and, as such, has first hand experience of clients needs as well as Online projects execution. Having spearheaded the deployment of Online methodologies across the group for several years now he is thus is a key witness of how the growth of Online is changing the face of research.
To advance the presentation slides use the space bar. To exit use Esc
Engaging Respondentsto drive the Quality of Online Research
15 March 2007Sebastien Janini
3
Survey experience and data quality
Towards a new survey interface
Building a relationship with the forgotten stakeholders
Delivering the value exchange promise
Engaging Respondentsto drive the Quality of Online Research
What’s new?
USA: 68
67
433
56
75
50
51
63
51
3530
5424
26
4764
42
24
36
38
46
26
14
2
15
20
76
5219
40
20
15
82
33
USA: 68
67
433
56
75
50
51
63
51
3530
5424
26
4764
42
24
36
38
46
26
14
2
15
20
76
5219
40
20
15
82
33
Internet penetration is ever growing
but is it the only gauge of quality?
Source: Internetworldstats.com & TNS – 2005 Users in %
6
A nomenclature of error types
Total Error
Random Sampling Error Non Sampling Error
Non Response Error Response Error
Researcher Errors Interviewer Errors Respondent Errors
• SurrogateInformation Error
• Measurement Error• Population Definition Error
• Sampling Frame Error• Data Analysis Error
• Questioning Error• Recording Error• Cheating Error
• Inability Error• Unwillingness Error
• Cheating Error (embellishment, numbing down)
Source: Naresh K. Malhotra, Marketing Research
Can the survey execution impact respondent experience and data quality?
Motivating respondents, Motivating respondents, generating curiosity and interestgenerating curiosity and interest
Dynamic interface to establish Dynamic interface to establish connectivity with respondentsconnectivity with respondents
The ‘practitioner’ point of view
experienceand data quality
survey
Online survey design optimisationDoes this look right?
Appearance and formatting of Interactive surveys …
Drop Down Box vs
Radio Buttons
… will affect response rate and response values
Respondents tend to opt in favour of going ‘higher’ in the drop down box response format than they did when using radio buttons
In some regions Language itself can be In some regions Language itself can be an issuean issue
“ 火 "
““Popular”Popular”
oror
““Hot”Hot”
ChinaChina
““Angry”Angry”
TaiwanTaiwan
Version A
BrandA
BrandB
BrandC
BrandD
BrandE
Version B
BrandA
BrandB
BrandC
BrandD
BrandE
BrandA
BrandB
BrandC
BrandD
BrandE
BrandA
BrandB
BrandC
BrandD
BrandE
Full grid versus split grid
Brand A
service
Version A: Full grid
16%22% 25%
11% 11%
Creative A dynamic brand Offer fashionable andinnovative products
Offer promotion or cashdiscount frequently
It’s my ideal brand
16% 15% 12% 13%21%
Reliable and trustworthy Produce high-quality cars Offer value-for-moneyproducts
High-tech. equipment Products are widely favored
13% 14% 14% 16% 16%
Engineering better Offer high-quality after-sales Has vision TV ad is reliable Customer oriented
Version B: Split gridSignificantly different
20%28% 27%
24% 24%
37%29%
17%24%
22%
19% 17% 15% 22%33%
Full grid versus split grid
Large vs Small text boxes
Large text boxes
Print Ad
Small text boxes
Print Ad
Q. What if anything do you like / dislike about this product?
Means: 10.46 8.51 10.85 11.13
Significantly different
Likes Dislikes
16% 16%27% 32%
56%68%
51%49%
8% 6% 4% 4%11%14%11%
21%
Large box Small box Large box Small box
21+ characters
11-20 characters
1-10 characters
Don't know / No answer
Large vs Small text boxes
Representing brands as text, logos or pack shots
18
Prompted Awareness by prompt type
0 20 40 60 80 100
Colgate
Aquafresh
Sensodyne
Macleans
Crest
Signal
Mentadent P
Oral B
Thera-med
Euthymol
Aim
Verbal
Logo
Pack
Little difference seen for major
brands
Lack of clarity of
pack shots reduces
awarenessDisastrously so!
But logo and pack can increase
awareness for smaller brands
19
The effect of different prompt types cannot be ignored.
Colgate Sensodyne Signal Euthymol Aim
Recommended by dentistsHigh quality
For sensitive teeth
Fresh tasting
Suitable for every day
Has a minty taste
Keeps teeth white
Keeps breath fresh
Product attributes
No significant difference between text logo and pack shot
At least one significant difference between text logo and pack shot
20
The effect of different prompt types cannot be ignored.
Colgate Sensodyne Signal Euthymol Aim
Modern and up to date
Good value for money
An upmarket brand
Easily recognisable
Attractive packaging
Caring
Eco-friendly
Used by younger people
Used by older people
Image attributes
No significant difference between text logo and pack shot
At least one significant difference between text logo and pack shot
21
Sensodyne
00.10.20.30.40.5
0.60.70.80.9
1
Rec
omm
ende
d by
dentis
ts
Hig
h qual
ity
For s
ensi
tive
teet
h
Fre
sh T
astin
g
Suita
ble
for e
very
day
Has
a m
inty
tast
e
Kee
ps te
eth w
hite
Kee
ps bre
ath fr
esh
Verbal Logo Pack
Product qualities
The pack detracts from product attributes
The pack detracts from product attributes
22
Aim
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Rec
omm
ended
by d
entis
ts
Hig
h qual
ity
For s
ensitiv
e te
eth
Fre
sh T
astin
g
Suita
ble fo
r eve
ryday
Has
a m
inty
tast
e
Kee
ps te
eth w
hite
Kee
ps b
reat
h fres
h
Verbal Logo Pack
Product qualities
The pack provides product cues.
The pack provides product cues.
Prompted Awareness is 3%!
23
You will get differences in awareness and image measurements with different forms of prompt
Packs and logos are designed to convey positionings and images and respondents do pick up on these in the interview
The effect can be to reduce association with claims that are not propounded in the logo or pack, particularly with bigger brands. Or it can be to reinforce specific product/image positions that are suggested by the logo or pack, particularly for lesser known brands
Respondents will answer based on the logo and, particularly, the pack alone, despite having no knowledge of the brand.
There is no right or wrong form of prompt, but different forms of prompt measure different things
A text prompt relies on what is in the consumer’s mind alone – the equivalent of writing the shopping list.
A logo prompt provides some further product or image cues
The pack provides (usually) even more specific cues – equivalent to standing in front of the facing in the supermarket.
If you change the form of prompt in a tracking study, you will lose continuity.
Representing brands as text, logos or pack shots
Benefits of best practices
The Internet and the Online data collection medium have made it possible for a wider range of businesses to have instant access to a huge amount of information at low cost
Finding information is NOT what Market Research is about
It is all about knowing who to survey and what the sample represents, how to ask the questions and what the biases in the method are, what the data collected mean and what it does not mean, how to analyse that data and how far it can be extrapolated
Discipline, structure and method in the research design are critical
Otherwise the researcher only ends up ‘finding stuff’
Training and Research on Research are key to develop a culture of best practices
Providing structure to surveys doesn’t necessarily mean designing ‘boring’ surveys.
We still need to push for a better look and feel, and new interface solutions
a new survey interface
Towards
Drag & Drop
Page Turner
29
Enhancing the survey process …
Rich stimulus
Rich data
Deeper consumer insights
Busy retail
environment …
30
… by using the visual language of the respondents
Stimulate
respondents
Engage
consumers
31
The convergence of digital media
32
33
A new interface
Bring in entertainment …
Movement away from conventional research to greater involvement / dialogue
A hybrid quantitative and qualitative approach
Scale brings rational intelligence
Open chat brings emotional intelligence
Mass marketing near an end, same dynamics for research
Clients looking for greater 'connection' or ‘engagement’ with their target
market/ audience
Significant trend and opportunity for insight communities / co-creation
'Death of distance‘ – or moribund
New software tools provide ability to Engage, Stimulate and Capture
consumer preferences Faster and more Accurately
with the forgotten stakeholders
Building a relationship
Access panel managementSuperior Panel Management
Response
Rates
Contact Controls
Usage Controls
Positive Panellist
Experience
37
Each survey is the face of the panel/research company
What does the survey invite say about you?
How do you treat respondents when they providewrong / incomplete response? Are we stillthinking ‘permission’ rather than ‘interruption’?
How motivating is the incentive scheme? What does it say about how you value your respondent?
Are you honest with survey length? Is there a progress bar?
Do you ensure respondent-friendly download times?
Are you working toward a more enjoyable & less intrusive interviewing experience? A respondent-friendly survey for better respondent engagement?
Building a relationship
38
Panel management rules tell a lot about how you value your panellists
What are reasonable recontact rules?
What do you do to maximise cooperation rates, to promote loyalty and tenure?
Is the look and feel of your surveys consistent? Are you actually building a brand panellists can recognise and engage with?
Are you maximising efficiencies (and minimising respondents frustration) via targeted sampling?
Are there any control for over-usage of hard to reach target groups?
Is there a policy of minimum fieldwork period length?
Building a relationship
39
Panel/research company need to build a relationship with panellists outside project work
An ‘Active’ management of panellists, outside project work, is critical to maximise engagement. It is about building a relationship (continual dialogue, feedback process, celebration of birthday dates)
What do you do to build and maintain trust with respondents?
Is there anything we can learn from how stores manage their loyalty card holders?
How about running employee satisfaction studies with the panellists? Tracking the results and setting targets for ‘commitment levels’
Building a relationship
the value exchange
promise
delivering
41
What do respondents want?
Passing some ‘value’ back to the
respondent
Concerns over…
- Growing individualism - Rejection of ‘your opinion counts’…what’s in it for me? - Privacy protection - Time - Over-contacting
Respondent co-operation rates are low and falling in many
markets…
42
Respondent engagement is a two way stream
Enhancing dialogue with respondents (less intrusive, more enjoyable)
Closer connection with respondents outside projects
Passing value back (incentives, feedback)
Asking respondents input into the panel management rules
Building a Brand
Developing ‘employee’ satisfaction management programmes
Quality controls embedded in the survey
Checking interview duration
Sense checking responses
Systematically picking up on patterned responding
Inserting checks/trap questions
Purging of non-responsive panelists
Online responses are faster, deeper, and more considered
Proven reliability of panel results Test retest Validation studies
People do respond differently online more considered responses More candid and honest Sensitive subjects More critical Richer open ended responses
Online
Better Research
Outcomes
Page 44
Thank you!