Existing and new approaches for analysing data from Check All That Apply questions
-
Upload
compusense-inc -
Category
Science
-
view
119 -
download
5
description
Transcript of Existing and new approaches for analysing data from Check All That Apply questions
![Page 1: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/1.jpg)
Existing and new approaches for the analysis of CATA data
Michael Meyners Procter & Gamble B. Thomas Carr Carr Consulting John C. Castura Compusense Inc.
![Page 2: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/2.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
![Page 3: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/3.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
![Page 4: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/4.jpg)
Are 2 products different?
Sign Test (exact)
McNemar’s Test (approximation)
Selected Not Selected
Selected a b
Not Selected c d
Product B
Pro
du
ct A
![Page 5: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/5.jpg)
Are 3 (or more) products different?
Cochran’s Q Test (approximation)
extends McNemar’s Test
![Page 6: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/6.jpg)
Testing strategy
Attributes Products Test 1 All 1 2
Cochran’s Q Test Sign Test
Significant?
![Page 7: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/7.jpg)
Testing strategy
Attributes Products Test All All 1 All 1 2
? Global Test ? Cochran’s Q Test Sign Test
Significant?
Significant?
![Page 8: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/8.jpg)
Global Test
Test Statistic: Sum of 𝑄 statistics
Approximate Test
Chi-square distribution formed using property of additivity
Problem: null distribution incorrect
(chi-square distributions are not independent)
![Page 9: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/9.jpg)
Global Test
Solution:
Randomization test (quasi-exact):
null distribution is valid because it is based on repeated and appropriate reshufflings of observed data
![Page 10: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/10.jpg)
Testing strategy
Attributes Products Test All All 1 All 1 2
Randomization Test Cochran’s Q Test Sign Test
Significant?
Significant?
![Page 11: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/11.jpg)
With CATA data, can we…
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
![Page 12: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/12.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
![Page 13: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/13.jpg)
1 2 3 … Ideal Total
Fresh 89 70 71 … 142 642
Warm 6 7 4 … 90 135
Crusty 24 4 32 … 47 193
Soft 76 95 75 93 570
Sweet 102 18 30 85 466
… … … … … ... …
Total 2169 1144 1539 … 2579
Contingency table
![Page 14: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/14.jpg)
Bar charts
![Page 15: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/15.jpg)
Problem: This is a mess.
We need a visual summary of this data.
Bar charts
![Page 16: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/16.jpg)
Correspondence Analysis
Reduces dimensionality of contingency tables (Think: PCA for ordinary data)
Attributes
Pro
du
cts
![Page 17: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/17.jpg)
Correspondence Analysis
Could use…
Chi-square distances
Sensitive to attributes with low counts
Hellinger distances
Low counts not a problem
![Page 18: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/18.jpg)
Correspondence Analysis
Sweet
2
4
3
1
5 6
Molasses
Exciting
Tempting
Satisfying
Tasteful Fullness Comfort
Dense
Firm
Moist Warm
Homemade
Fresh Chewy Traditional
Soft Springy Wheat
Brown
Healthy
Nutritious
Crusty
Texture
Coarse
Sesame
Rustic Nutty
Crunchy
Grainy Seeds
Component 1 (79.2%)
Co
mp
on
ent
2 (
11
.9%
)
![Page 19: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/19.jpg)
Component 1 (79.2%)
2 3
1
5 6 Traditional
Soft Springy Brown
Nutty
Crunchy Grainy
Seeds
4
![Page 20: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/20.jpg)
Correspondence Analysis
Can only look at 2 (or 3) dimensions at once Problem: True relationships between products and attributes might be obscured.
![Page 21: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/21.jpg)
Multidimensional Alignment (MDA)
Solution: Look at each product individually.
![Page 22: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/22.jpg)
![Page 23: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/23.jpg)
Positive association with these attributes
Negative association with these attributes
![Page 24: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/24.jpg)
With CATA data, can we…
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
![Page 25: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/25.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
![Page 26: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/26.jpg)
φ-coefficient
Measures correlation between 2 binary attributes
![Page 27: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/27.jpg)
ClassicalMDS on matrix of 𝟏 − 𝝋 distances
Fresh
Warm Crusty
Soft
Moist
Nutty
Tempting Sweet
Dense
Texture
Wheat
Exciting
Crunchy
Homemade
Seeds
Satisfying
Coarse
Nutritious
Molasses
Tasteful Rustic
Traditional
Springy
Sesame
Firm
Fullness
Healthy
Grainy
Comfort
Brown
Chewy
![Page 28: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/28.jpg)
ClassicalMDS on matrix of 𝟏 − 𝝋 distances
Fresh
Warm Crusty
Soft
Moist
Nutty
Tempting Sweet
Dense
Texture
Wheat
Exciting
Crunchy
Homemade
Seeds
Satisfying
Coarse
Nutritious
Molasses
Tasteful Rustic
Traditional
Springy
Sesame
Firm
Fullness
Healthy
Grainy
Comfort
Brown
Chewy
![Page 29: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/29.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
![Page 30: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/30.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
![Page 31: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/31.jpg)
Penalty-Lift Analysis
Problem: What is the relationship between what consumers perceive and their hedonic response? Solution: Look at elicitations of attributes and association with liking
![Page 32: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/32.jpg)
change in liking (attribute elicited vs. not elicited)
![Page 33: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/33.jpg)
change in liking (attribute elicited vs. not elicited)
![Page 34: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/34.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
![Page 35: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/35.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
![Page 36: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/36.jpg)
Comparison with Ideal
Problem: How to modify a product to delight consumers? Solution 1: Close gap between product and ideal (accounting for natural variability!)
![Page 37: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/37.jpg)
Difference in elicitation rates
Product – Ideal
![Page 38: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/38.jpg)
Difference in elicitation rates
Product – Ideal
![Page 39: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/39.jpg)
Penalty Analysis
Problem: How to modify a product to delight consumers? Solution 2: Focus on the most relevant attributes for consumers’ hedonic responses
![Page 40: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/40.jpg)
% consumers
Higher Relevance for Many
Consumers
+2
+1
0
10 30 50 40 20
Penalty analysis based on “must have” attributes
Liki
ng
Fresh Warm
Crusty
Soft
Moist
Chewy
Nutty
Tempting
Sweet
Grainy
Dense
Texture
Healthy
Exciting
Crunchy
Homemade
Brown
Seeds
Satisfying
Coarse
Fullness
Nutritious
Molasses
Tasteful
Rustic Traditional
Springy
Sesame
Wheat
Comfort
![Page 41: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/41.jpg)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
Yes!
![Page 42: Existing and new approaches for analysing data from Check All That Apply questions](https://reader031.fdocuments.in/reader031/viewer/2022020105/549e6e6bb479597e208b4741/html5/thumbnails/42.jpg)
Thank you for your attention!
For more information see the following publication and references therein:
Meyners, M., Castura, J.C., Carr, B.T. (2013). Existing and new approaches for the analysis of CATA data. Food Quality and Preference, 30, 309-319.
B. Thomas Carr Carr Consulting
Michael Meyners Procter & Gamble
John C. Castura Compusense Inc.