What, when and why to use these question types

Divider Page Example with Picture Mapping Standard Question Types onto Visual

Alternatives

Single Choice

3

STANDARD   VISUAL  

Single Choice Grid

4

VISUAL  

STANDARD  

Single Choice Grid #2

5

VISUAL  (Magnetic  Board)  

STANDARD  

Single Choice Grid #3

6

VISUAL  (Card  Sort)  

STANDARD  

Multiple Choice

7

VISUAL  

*  Combination  of  options  with  images  and  no  images  for  demo  purposes.  

STANDARD  

Multi Choice Grid

8

VISUAL  

STANDARD  

Multiple Choice Grid #2

9

VISUAL  (Magnetic  board)  

STANDARD  

Multiple Choice Grid #3

10

Rank Order

11

VISUAL  STANDARD  

Scale Question

12

VISUAL  (Scale    Slider)  

STANDARD  

Numeric Response Question

13

VISUAL  (Numeric  Slider)  

STANDARD  

Allocation (Constant Sum)

14

VISUAL  (Allocation  Slider)  

STANDARD  

Geographic Location

15

VISUAL  STANDARD  

Example  text  Go  ahead  and  replace  it  with  your  own  text.  This  is  an  example  text.    

 

Example  text  Go  ahead  and  replace  it  with  your  own  text.  This  is  an  example  text.    

 

Example  text  Go  ahead  and  replace  it  with  your  own  text.  This  is  an  example  text.    

 

16

Example  text  Go  ahead  and  replace  it  with  your  own  text.  This  is  an  example  text.    

 

Example  text  Go  ahead  and  replace  it  with  your  own  text.  This  is  an  example  text.    

 

Example  text  Go  ahead  and  replace  it  with  your  own  text.  This  is  an  example  text.    

 

16

Visual Questions that do what standard

Page Turner

17

Media Player

18

Highlighter

19

Click-map

20

So how do these visual questions effect respondents?

engagement and data quality

21

We will go deeper on these question types

22

Engagement measures: Visual Question types vs. Flat Grid Questions

Multi-­‐choice  (MC)   Single  choice  (SC)  

Magnetic  Board  

Visual  Grids  (US)  

Visual  Grids  

(Canada)  

Scale  Slider  

Visual  Grid  (US)  

Visual  Grid  

(Canada)  

Easier  to  complete   =   =   =   =  Fun  to  complete   =   =  More  enjoyable  than  most  

=   =  

The  Visual  Grid  and  Multi  Choice  Magnetic  Board  question  types  were  significantly  more  engaging  than  flat  grid  questions.    Scale  Sliders  were  equal.  

Significantly  higher  rating,  95%  CI  

Equivalent  rating,  95%  CI  =  

Prior studies have also shown greater engagement

Papers  presented  at  CASRO  and  ESOMAR  showed  a  similar  pattern  of  increased  respondent  engagement  with  visual  question  types,  compared  to  flat  question  types.  

Sliders and Visual Grids

25

Visual  Grid  

Scale  Slider  

Flat  Single  Choice  Grid  

Sliders and Visual Grids

Sliders  and  Visual  Grids  are  used  for  rating  exercises  like  attitudinal  statements  and  brand  ratings.    Visual  Grids  (both  single  and  multi  choice)  are  rated  as  being  more  fun,  easier  and  more  enjoyable  than  radio  button  grids.        Both  Visual  Grids  and  Sliders  encourage  people  to  use  more  of  the  scale,  and  less  of  the  midpoint,  than  flat  radio  buttons,  on  a  10  point  scale  anchored  verbally  at  the  ends.        On  verbally  anchored  scales,  much  less  variation  is  observed,  suggesting  the  verbal  anchoring  makes  an  important  difference.  

Comparison of how 10 point brand association scales are used for Flat Single Choice Grid (radio buttons), Visual Grid and Slider Scales

Scale  Points  

Percent  o

f  Respo

ndents  

US  study   Canadian  study  

Scale  Points  

In  the  US  study,  the  differences  seem  to  driven  by  a  more  varied  use  of  the  scale  points  by  people  answering  the  visual    

produced  a  more  normal  distribution  

Note:  different  brands  were  rated  in  each  country.      These  numbers  represent  the  combination  of  the  answers  for  all  brands  

Comparison of how 5 point verbally anchored scales are used for Flat Grids, Visual Grid and Slider Scales With  this  verbally  anchored  scale  we  do  not  see  any  important  variation  between  question  types,  

The  smaller  number  of  scale  points  may  also  influence  it.  

[RICH  MEDIA w  or  have  heard  about  them,  either  by  clicking  on  the  column  under  the  rating  or  by  dragging  the  checkmark  to  the  column  you  choose.  [FLAT u  know  or  have  heard  about  them,  by  clicking  on  the  column  under  the  rating  you  choose.  

Note:  different  brands  were  rated  in  each  country.      These  numbers  represent  the  combination  of  the  answers  for  all  brands  

US  study   Canadian  study  

Card Sort

29

with the card sort than with radio buttons

52   46  

0

10

20

30

40

50

60

Card  Sort Radio  Buttons

3  

11  

0

2

4

6

8

10

12

Card  Sort Radio  Buttons

More  people  doing  the  card  sort  

responses  

People  doing  the  radio  button  survey  were  more  likely  to  

statements  

%  with  zero  DK  

%  DK  to  all  

Verbally Anchored Scale--Trustworthiness

With this verbally anchored rating scale we see slightly greater use of the midpoint and less use of the top box, with the radio button vs. the card sort

0

5

10

15

20

25

30

35

40

45

50

Completely Very Fairly Not  Very Not  at  all

Card  Sort Radio  Button

We  saw  no  difference  in  the  amount  of  time  required  to  do  the  card  sort,  compared  to  the  radio  button  

Card Sort Usage Conclusions

Attitude  statements  are  good  candidates  for  use  with  card  sorts.        Ratings  of    single  items  is  also  a  good  use,  as  the  data  appears  to  be  more  varied.        Card  sorts  also  provide  a  welcome  change  of  the  type  of  task  respondents  are  asked  to  do.    

Multi Choice Magnetic Boards

Multi Choice Magnetic Boards

People  enjoy  the  magnetic  board  exercise,  rating  it  higher  on  

button      While  conceptually  similar  to  a  visual  grid  or  radio  button  grid,  we  find  that  the  magnetic  board  question  results  in  different,  yet  similar  results.        We  conducted  a  head  to  head  test  of  a  magnetic  board  question  vs.  a  radio  button  and  a  visual  grid  design,  using  three  different  arms  of  the  study,  each  of  whom  used  a  different  question  type.        

0%

10%

20%

30%

40%

50%

60%

70%

80%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51

Visual Grid Radio Button

Magnetic Board

Comparison of Multi-choice (binary) Brand Associations Using MC Grid (radio button), Visual Grid and Magnetic Boards

[RICH  MEDIA VISUAL  GRID  &  SLIDER]  Here  is  a  list  of  words  and  phrases  that  might  describe  these  stores,  to  varying  degrees.    Please  indicate  how  closely  you  associate  each  word  or  phrase  with  each  store,  by  either  clicking  on  the  column  under  the  rating  or  by  dragging  the  checkmark  to  the  column  you  choose.  [FLAT RADIO  BUTTON  GRID]  Here  is  a  list  of  words  and  phrases  that  

might  describe  these  stores,  to  varying  degrees.    Please  indicate  how  closely  you  associate  each  word  or  phrase  with  each  store,  by  clicking  on  the  column  under  the  rating  you  choose.    

When  people  used  the  Magnetic  Board,  the  pattern  of  attributes  chosen  was  very  similar  to  that  observed  with  the  grids,  but  there  were  roughly  half  the  number  of  attributes  chosen.    

 These  numbers  represent  the  combination  of  the  answers  for  all  brands  

Multi Choice Magnetic Boards: time spent and explanatory power

37

People  spent  as  long  on  the  magnetic  type  exercise  as  they  did  on  the  visual  grid even  those  they  ended  up  choosing  half  the  number  of  attributes    Interestingly,  when  we  modelled  the  strength  of  the  attributes  ability  to  predict  overall  liking  of  the  brand,  we  found  that  the  visual  grid  and  magnetic  type  question  types  had  the  same  level  of  explanatory  power even  though  there  were  half  the  number  of  attributes  included.          

Multi Choice Magnetic Boards: Conclusions

38

Multi  choice  magnetic  boards  are  engaging,  but  people  respond  to  them  differently.    choice  grids  questions    in  a  tracker.        But  do  think  of  magnetic  boards  for  things  like  monadic  brand  association  exercises,  shopping  exercises  or  the  like.                  

Highlighter and Click Map: Unique data?

39

While  the  highlighter  can  obviously  be  used  in  in-­‐person  interviews  in  the  real  world,  there  is  no  other  equivalent  way  of  capturing  this  data.    Click  maps,  however,  have  been  compared  to  eye  tracking,  a  relatively  costly  and  laborious  process.            

Research on Click Maps Vs. Eye Tracking

40

When looking at total clicks versus eye movements, results are similar

Click  Map  Online  

Note: Heat mapping shows all clicks (up to 3 clicks per person) and all views (6 seconds). Online: Please look at the package below and click on the first three things that catch your attention. CLT: Please look at the products on the screen as you would

High  

Low  

Traditional  Central    Location  Eye  Tracking  

Highlighter and Click Map Conclusions

41

The  data  is  qualitative,  but  the  reporting  is  very  clear    

Click  maps  can  replace  eye  tracking,  in  some  uses.      

When  dealing  with  multiple  languages,  know  that  the  highlighter  results  will  need  to  be  analysed  separately  

Conclusions

So what do we make of these question types?

We  have  seen  

We  see  that  visual  question  types  tend  to  be  more  engaging    We  also  see  that  they  gather  somewhat  different  data:  data  

   But  in  the  case  of  something  like  magnetic  type,  we  see  that  visual  questions  can  capture  data  that  is,  in  some  ways,  very  different  .        And  in  the  case  of  something  like  a  click  map,  we  find  it  can  be  very  similar  to  something  like  eye  tracking.  

But the interesting question is: how do you see them being used? How might you use them in new and unique ways?

44