Conjoint Analysis: past, present and issues

Gilbert Saporta CEDRIC- CNAM,

292 rue Saint Martin, F-75003 Paris

gilbert.saporta@cnam.fr http://cedric.cnam.fr/~saporta

Bari,July 2015 1

Outline 1. Conjoint analysis basics

a. Historical background b. An individual compensatory utility model c. Choice and market share simulation

2. Data collection and analysis a. From full profiles to adaptive choice-based conjoint b. Models: OLS, monotone regression, multinomial logit c. Which conjoint method? d. Software implementations

3. Issues a. About the “none” option b. Partial profiles and answers c. Internal expertise or third party ?

4. Conclusion

Bari,July 2015 2

1. Conjoint Analysis basics

• One of the most successful statistical techniques in market research

• Aims at quantifying how people make choices between products or services

• A complete survey methodology including data collection based on experimental designs, a data analysis phase with parameter estimation, a simulation phase

Bari,July 2015 3

Bari,July 2015 4

http://www.surveyanalytics.com/conjoint-analysis-example.html

• About 3000 papers per year (Google scholar)

• Not only marketing: health, education etc.

Bari,July 2015 5

1.a Historical background

Bari,July 2015 6

Journal of Marketing Research Vol. 8, No. 3 (Aug., 1971), pp. 355-363

• A monotonous regression applied to rank order data described by a full design

Bari,July 2015 7

Media-planner’s rankings of 40 ad.vehicle combinations

Kruskal’s monanova algorithm

Genealogy of conjoint measurement:

• Luce & Tukey 1964 , Debreu 1959, Von Neumann & Morgenstern 1947

• Conjoint measurement, as practised by mathematical psychologists, has primarily been concerned with the conditions under which there exist measurement scales for both the dependent and independent variables, given the order of the joint effects of the independent variables and a prespecified composition rule.

• The term conjoint analysis means decomposition into part-worth utilities or values of a set of individual evaluations of, or discrete choices from, a designed set of multi-attribute alternatives (Louviere 1988)

Jarmo Heinonen http://www.metodix.com/en/sisallys/01_menetelmat/02_metodiartikkelit/heinonen_conjoint_methods/kooste

Bari,July 2015 8

Bari,July 2015 9

1.b An individual compensatory model

• A product is defined by a combination of p attributes

• The perfect product is generally unrealistic like a car with high speed, comfort, security and low price

• A compensatory model: the consumer makes a “Trade off” between attributes by putting into balance advantages and inconveniences.

• Conjoint analysis decomposes preferences according to an additive utility model, specific to each interviewee Bari,July 2015 10

• A product defined by a combination of levels (i,j,k,l, ..) of p attributes will have a global utility equal to ai+bj+ck+ …

• Coefficients or part-worth utilities are different for each respondent

• N additive models without interaction are fitted: no global model

• If all combinations are feasible: products and independent coefficients

Bari,July 2015 11

1

p

jj

m=

∏1

p

ii

m p=

−∑

1.c Choice and market share simulation

• Let 3 competing products • For each respondent i : Ui

1 ; Ui2 ; Ui

3

• Several models for respondent choice: – Maximal utility (deterministic) – Probabilities proportional to Ui

j (Bradley-Terry-Luce )

– Probabilities proportional to exp (Uij)

(« logit »)

Bari,July 2015 12

• A minimal utility level may be necessary. Some « poor » products will never be chosen.

• Solutions: – Ask a « will buy » question for each submitted

product, hence the purchase intention, plotted against the mean utility, averaged over all respondents

Bari,July 2015 13

Logistic fit

score

Purc

hase

inte

ntio

n

4 5 6 7 8 9 10 11 120

0.1

0.2

0.3

0.4

– Use a choice based conjoint design with a «none » alternative

Bari,July 2015 14

© Ipsos Market Quest

2. Data collection and analysis

• First implementations were done with the full profiles method

• Since is generally too large, one or several subsets of K products are submitted to a sample of consumers

Bari,July 2015 15

1

p

jj

m=

∏

2.a From full profiles to ACBC

• Example « Frozen entrees » (Kuhfeld, 2009)

• p= 4 features

Bari,July 2015 16

1

54p

jj

m=

=∏

Bari,July 2015 17

Bari,July 2015 18

• Attribute importance – If all products are feasible , the total range (utility

of the best – utility of the worst) is equal to the sum of part-worth utilities

– Importance defined as the % of utility range

Bari,July 2015 19

• Ranking or ratings? – Each of the 18 products is presented on a card and

consumers are asked to sort them (preference order). A rather difficult task. So why not rate the products?

– Rating expresses intensity of preferences, but: no comparison between products, problems with comparability of scales across respondents, risk of ties

– Ranking usually preferred – Often processed as a continuous variable! See

later

Bari,July 2015 20

• About the design – Frozen entrees: one third of the complete design – Orthogonal design: all effects may be estimated

without confounding – All pairs of attributes have balanced levels – Non orthogonal designs may be used to decrease

the number of products • 8 products is the minimal set in frozen entrees

example, since there are 7 part-worth utilities to be estimated: (3-1) + (3-1) + (3-1) + (2-1)

Bari,July 2015 21

• A few useful designs for 2 levels attributes – Factorial fractional designs – Plackett & Burman

• Latin and graeco-latin squares for attributes with the same number of levels

• D-optimal designs otherwise

Bari,July 2015 22

Bari,July 2015 23

A B C D E F G

---------------------------

1 1 1 1 2 2 2 1

2 2 1 1 1 1 2 2

3 1 2 1 1 2 1 2

4 2 2 1 2 1 1 1

5 1 1 2 2 1 1 2

6 2 1 2 1 2 1 1

7 1 2 2 1 1 2 1

8 2 2 2 2 2 2 2

L8 27 (Taguchi) or 27-4 (Box-Hunter)

Bari,July 2015 24

Plackett- Burman design L12 211 A B C D E F G H I J K 1 2 1 2 2 2 1 1 1 2 1 1 1 2 1 2 2 2 1 1 1 2 2 1 1 2 1 2 2 2 1 1 1 1 2 1 1 2 1 2 2 2 1 1 1 1 2 1 1 2 1 2 2 2 1 1 1 1 2 1 1 2 1 2 2 2 2 1 1 1 2 1 1 2 1 2 2 2 2 1 1 1 2 1 1 2 1 2 2 2 2 1 1 1 2 1 1 2 1 1 2 2 2 1 1 1 2 1 1 2 2 1 2 2 2 1 1 1 2 1 1 2 2 2 2 2 2 2 2 2 2 2

• Ranking a large number of products is a burden!

• Empirical bound for the number of comparisons : K ≤ 16 profiles

• Paired comparisons, choice based and (or) adaptive designs are often preferred to full profile designs

Bari,July 2015 25

ACA

• Developped by Sawtooth Software, ACA stands for adaptive conjoint analysis

• Success linked to the development of CAPI and CAWI

• Core of the method: a set of binary questions involving an increasing number of attributes, depending on the previous answers, until parameters (part-worth utilities) are estimated with enough precision, in a bayesian style

Bari,July 2015 26

• Prior importance and categories ordering are estimated through introductory questions like:

Bari,July 2015 27

Discrete Choice Models

• Instead of rating or ranking product concepts, respondents are shown several sets of products on the screen and asked to indicate which one they would choose.

• Also known as Choice Base conjoint or CBC

Bari,July 2015 28

29

Which of the following alternatives would you prefer, in

any?

• $ 225

• Parent controls

• Standard remote

• Medium height

• $ 275

• No parental controls

• Standard remote

• Medium height

• $ 300

• Parental controls

• Universal remote

• Large

I don’t like any of these alternatives

© Ipsos Market Quest

• Choice tasks are simpler than full profiles rankings, closer to real situations

• The set of choice questions is obtained by design of experiments techniques

• Adaptive versions of CBC have been proposed • However some authors consider that CBC is not

conjoint analysis; see Louviere et al. , 2010 Discrete Choice Experiments Are Not Conjoint Analysis, Journal of Choice Modelling, 3, 3

Bari,July 2015 30

2.b Estimation • OLS y vector of ranks min|| y - Xb || 2 y = Xb + e

Bari,July 2015 31

1

1 12

2 2

12 12

3

01 | 01 | 01 | ..... |10001 | 01 |10 | ..... | 010 ... ....10 |10 |10 | ..... |100

y ey e

y e

αα

ξ

= +

• Specificities – Model not of full rank: constraints on utility

coefficients. The most popular constraint: α1 + α2 + α3 = 0 – Only differences α1 - α2 are estimable – Criticism to OLS : ranks are not quantitative

variables

• Monotonous regression – fit T(y) instead of y where T is a monotonous

transformation of ranks: – minimize || T(y) - Xb || 2 over T and b

Bari,July 2015 32

Bari,July 2015 33

• However: – High overfitting risk

– OLS more robust

Bari,July 2015 34

• Goodness of fit – Measure the agreement between initial ordering and

the estimated one: –R2 or Kendall’s τ –Minimum value : a common practice discards

respondents with low R2, but:

Incoherence or ill-posed (no trade-off) problem? eg: -mobile phone, whatever the price - garbage bags, whatever colour, texture, closing system

Bari,July 2015 35

• Multinomial logit model for choice based experiments

Bari,July 2015 36

The multinomial logit model assumes that the probability that an individual will choose one of the m alternatives, ci, from choice set C is

where xi is a vector of coded attributes and β is a vector of unknown attribute parameters (part-worth utilities) . U(ci) = xi β is the utility for alternative ci, which is a linear function of the attributes.

Kuhfeld,2010

( )

( ) ( )1 1

exp ( ) exp( )( / )exp ( ) exp

i ii m m

j jj j

U cP c C

U c= =

= =

∑ ∑x β

x β

• Case study: launch of a public transportation pass for young people (12-26) of Paris region

• 1200 respondents • 5 attributes:

• Duration (2), price (4), zone-options (4), bonus card (2), communication (2)

Bari,July 2015 37

• A specific model for binary choice – Choice between pairs of products – Example : 5 attributes

• Product A x’(A)=10 1000 0001 10 10 • Product B x’(B)=10 0100 0100 01 01

• b=(b1,b2 ….,b14) : utilities vector • Scores s(A)=x’(A)b s(B)=x’(B)b

– A is preferred to B if s(A)-s(B)>0 (x’(A)-x’(B))b>0

Bari,July 2015 38

Bari,July 2015 39

• If n is the number of binary choices (« duels ») Xb y

1

..

..

.

00 1-100 0-101 1-1 1-1

q

b

b

+ − + −

• X may be obtained through D-optimal design • Estimation of b

– Logit model (logistic regression) and maximum likelihood estimates seems appropriate but many degeneracies: perfect separation when consumers are rational!

– Fisher’s linear discriminant function (or OLS regression) works in all cases

Bari,July 2015 40

2.c Which conjoint method?

• Subjective criteria

Bari,July 2015 41 Furlan, Corradetti, 2005

• Objective criteria –Number of attributes, number of

levels –Survey mode , material to be

presented

Bari,July 2015 42

http://www.sawtoothsoftware.com/index.php?option=com_content&view=article&id=658

Bari,July 2015 43

Bari,July 2015 44

Bari,July 2015 45

• Could be biased : CBC is the flagship product of Sawtooth Software…

Bari,July 2015 46

2.d Software implementations

• General purpose softwares

Bari,July 2015 47

• Specialized softwares

• Free R package

Bari,July 2015 48

3. A few issues

• Standard issues: – Influence of level choice on attribute

importance like price – Main effects only – Beware of means! Perform segmentations

on utilities to identify homogenous groups of respondents

Bari,July 2015 49

3.a The « no choice » issue

• Elrod, Louviere and Krishnakumar (1992) specify the no choice as another alternative with the attributes equal to zero and determine the choice between the products and the option ”zero” by comparing their utilities.

• Highly arguable!

Bari,July 2015 50

• Ohannessian & Saporta, 2008 proposed a solution inspired by the censored regression models (tobit models) that suppose a change of the dependent variable from a certain threshold. A comparison between the utilities remains, but it only takes place between the products utilities, because the ”zero” option is not described by an utility.

• Another explanation of the « no choice » was also proposed: conflict

Bari,July 2015 51

Bari,July 2015 52

• Refusal or conflict when utilities are too close

3.b Incomplete rankings

• In classical full-profiles interviews, respondents may rank only their top choices , or are only reliable for them.

• Simulation studies tends to prove that ranking half of the scenarios is enough to estimate utilities. (Benammou & al, 2003)

Bari,July 2015 53

3.c Internal expertise or third parties?

• At least in the french market, Sawtooth Software’s products have a dominant position especially in market research companies (eg BVA, IPSOS, TNS Sofres)

• Loss of expertise by the end users and by consultants – Easier to use CBC than writing code lines in SAS or

R!

Bari,July 2015 54

Conclusion

• CA is a versatile technique, very useful to quantify consumer’s decisions

• Common features of various methods – a trade-off hypothesis – Computation of individual part-worth utilities – Market share simulation

• Neglected: Hierarchical Bayes, MaxDiff , or Best/Worst and a few others

Bari,July 2015 55

• Beyond market research studies: applications to new fields of human decision (medicine)

• Academic production still high • Only few tools used in companies

Bari,July 2015 56

Grazie per l'attenzione!

Bari,July 2015 57

Bari,July 2015 58

References • Benammou S., Harbi S., Saporta G. (2003), Sur l'utilisation de l'analyse

conjointe en cas de réponses incomplètes ou de non-réponses - Revue de Statistique Appliquée 51, 31-55.

• Furlan R., Corradetti R. (2005) An empirical comparison of conjoint analysis models on a same sample, Rivista di Statistica Applicata, 17,2, 141-158

• Green P.E., Rao V.R. (1971) Conjoint Measurement for Quantifying Judgmental Data , Journal of Marketing Research , 8, 3, 355-363

• Green P.E., Srinivasan V. (1990), Conjoint analysis in marketing: new developments with implications for research and practice, Journal of Marketing, 3-19

• Kuhfeld W. (2010), Marketing research methods in SAS, SAS 9.2 Edition, MR-2010

• Louviere J. J. (1988), Analyzing Decision Making – Metric Conjoint Analysis, Sage University Papers.

• Ohannessian S. , Saporta G. (2008) Zero option in conjoint analysis, A new specification of the indecision and the refusal., SIS'08, Univ. Calabria, Cosenza