Examining the “No-Choice” Option in Choice-Based Conjoint Analysis

Maggie Chwalek, Advisors: Dr. Greg Allenby and Dr. Roger Bailey

Research Thesis

Abstract

Choice-Based Conjoint analysis (CBC) is a method used to determine how individual

consumers value attributes of a product or service. The consumer selects the option he or she

would be most likely to purchase among those presented, which includes a “no-choice” option.

The “no-choice” option is an alternative within a CBC analysis that allows consumers to decide

against purchasing any of the presented options. This no-choice option is necessary in order to

correctly simulate real choices, but could be problematic if each consumer perceives this option

differently (Allenby, et. al. 1995). The goal of this research is to provide a recommendation for

how to improve the consistency of choices in CBC analysis. We employ the framework in

General Evaluability Theory (Hsee & Zhang 2010) to justify potential improvements to CBC.

We hypothesize that evaluability of the no-choice option can be improved by creating a

consistent view of the frontier of products available to the consumer. We will test this hypothesis

by sampling 1000 respondents randomly split into three different groups, with each group

receiving a different amount of information about the products in the market. A control group

will engage in standard CBC, the first treatment group will be shown the prices of products in the

market, and a second treatment group will be shown the price of products in relation to multiple

attributes of the product. The results show no significant differences in the propensity to select

the no-choice option between the three experimental groups. Fitting a Hierarchical Multinomial

Logit Model to the data, three attributes show significant differences in the upper level between

the control group and treatment group one. This suggests that participants in the treatment group

value products with those attributes more highly compared to participants in the control group.

Further investigation looked at qualified versus non-qualified survey participants and showed

that additional information about the market gave non-qualified participants a lower propensity

to select the no-choice option. We conclude that using proper screening techniques allows

standard Choice-Based Conjoint Analysis to be a robust tool for marketing researchers given the

results from qualified survey respondents.

Acknowledgements

Thank you to my research advisors, Dr. Greg Allenby and Dr. Roger Bailey, for guiding,

teaching, and mentoring me through this process. I have gained a great appreciation for

research thanks to you both.

Table of Contents

Abstract.....................................................................................................................................................2

Acknowledgements.................................................................................................................................3

List of Figures..........................................................................................................................................5Introduction.............................................................................................................................................6

Literature Review...................................................................................................................................6

Hypothesis.................................................................................................................................................8Methodology.............................................................................................................................................9

Data Analysis and Results..................................................................................................................11Discussion..............................................................................................................................................12

References..............................................................................................................................................13

Appendix................................................................................................................................................14

List of Figures

Figure 1. One of twelve choice tasks used in the CBC study.

Figure 2. Graph shown to participants in the first treatment group.

Figure 3. One of four graphs shown to participants to participants in the second treatment group.

Figure 4. Initial descriptive results for participants’ propensity to select the no-choice option

across experimental groups.

Figure 5. Statistical significance in δ7,2.

Figure 6. Non-qualified respondents’ propensity to select the no-choice option.

Introduction

Conjoint analysis is a method used to determine how individual consumers value

attributes of a product or service. This type of analysis is used often because it closely mimics

how customers make decisions in the market and can be very effective in predicting their price

sensitivity and purchase decisions. The “no-choice” option is an alternative presented within a

choice-based conjoint (CBC) analysis that allows consumers to decide against purchasing one of

the presented options in the survey. This option is necessary in order to correctly simulate real

choices, but could be problematic if each consumer perceives this option differently and the

results of the CBC study are affected (Allenby, et. al. 1995). This research examines the effect of

building a more consistent interpretation of the no-choice option across participants in a CBC

study.

This research thesis will begin by introducing the previous literature on conjoint analysis.

The next section will explain the variables and hypotheses of the research, followed by a

discussion of the methods that will be used to complete this research study. Finally, this paper

will include an analysis of the data collected during the study, results, and discussion.

Literature Review

Current research on the no-choice option in choice-based conjoint (CBC) analysis has

focused on redesigning the choice tasks as a two-step process to increase prediction accuracy and

estimation in the model. One article presents the idea of a dual response format in CBC. In the

first decision, consumers choose between alternatives without a no-choice option. Next,

consumers choose between the selected alternative and a no-choice option. The researchers argue

that this method is effective by taking advantage of the real life decision-making simulation

(including the no-choice option) while reducing bias in the results. The results of this research

noted that the dual response method did not perform significantly better than the traditional

method. This research gives us insight on the effects of restructuring CBC studies and the no-

choice option (Brazell, et. al. 2006).

Another paper has presented theoretical ideas of combining a traditional CBC method

with behavioral aspects to fully understand consumer preferences and decisions. The researcher

argues that consumers are not able to process all information given in order to calculate the

utility of a decision as well as the alternative. This creates the need for a behavioral aspect to

accurately understand consumers’ decisions and preferences. The researcher presents five ways

of incorporating a behavioral aspect into the traditional CBC method. She has valid arguments

from past research that support the improvement in traditional CBC, however no data was

collected using the methods she suggests. The research here can be applied to this project by

incorporating the behavioral economic and psychology guidance. (Caldwell 2015).

The research papers above share a common theme: they do not address the issue of

creating a coherent view of the frontier of products available to the consumer, thereby building a

more consistent interpretation of the no-choice option across participants in a CBC study. We

can use the research above by understanding how changing the structure of CBC affects the

outcome of a study.

Previous literature regarding the evaluability of an alternative plays an important role in

this research. The General Evaluability Theory (Hsee & Zhang 2010) addresses questions

surrounding value sensitivity and a consumer’s accurate prediction of his or her own as well as

others. The first proposition of GET states that evaluability depends on three independent

factors: mode, knowledge, and nature. Mode can either be single where only a single alternative

is presented, or joint, where two or more alternatives are compared. The research claims that

joint evaluation yields higher value sensitivity due to the individual’s ability to reference another

option. The second proposition states that if mode, knowledge, and nature are all of low

sensitivity, then the final outcome will also be of low sensitivity. Finally, the third proposition

says that people who are able to be highly value sensitive will overestimate the value sensitivity

of those in a low evaluability.

The research of Hsee and Zhang (2010) is important to this research topic because the

goal of this research is to make the no-choice option evaluable to consumers. The no-choice

option can be viewed as a combination of all other options in the market that are not represented

in the discrete choice task. This option is therefore difficult to compare to other options in the

choice task. By using the framework propositions for evaluability, Hsee and Zhang have

proposed, this research should be able create a consistent view of the frontier of products

available to the consumer. Defining the frontier of products in the market will be a combination

of previous knowledge of the product category, innate understanding of the product category,

and knowledge learned from the descriptions in the survey.

This literature review helps us to understand where the no-choice option has its strengths

and weaknesses. We can employ the framework in General Evaluability Theory (Hsee & Zhang

2010) to justify potential improvements to CBC. In addition, we can observe the effects of

restructuring a CBC study on results and build on previous research by applying similar methods

to changes in the no-choice option.

Hypothesis

We hypothesize that we can build a consistent interpretation of the no-option by giving

participants background information on the products in the market prior to participating in the

study’s choice tasks. We suspect that a one-dimension graph of the prices of products in the

market is an evaluable and appropriate display of the frontier of products and that the

participants who are shown this price graph will have more consistency in their choices than

standard CBC. Participants who are given information about the market beyond the price may or

may not have more consistency in their choices, as the amount of information may be too great

to use.

Methodology

The teeth whitening product market was used in the survey to test the effect of different

levels of information given to consumers on their choices in the survey. The following attributes

of teeth whitening products were used to create 12 choice tasks:

• Brand

• Form

• Time per treatment

• Number of treatments

• Time to results

• Percent peroxide

• Price

A choice task is used in CBC studies and typically displays three alternatives as well as the no-

choice option. The alternatives are configured from the attributes of the products selected. Each

attribute has multiple levels. For example, the form attribute has four levels: strips, pen, trays +

gel, and light technology. Each alternative receives only one level of each specific attribute.

Figure 1 is an example of a choice task used in the CBC study.

Participants were screened out of completing the choice tasks in the survey based on a

number of reasons. Reasons included if the participant is not involved in purchasing hygienic

products for his or her self or household, if he or she is not interested in purchasing teeth

whitening products, or if he or she has had or plans to have dental work on which teeth

whitening products will not be effective.

All qualified survey respondents were shown a glossary video prior to engaging in the

choice tasks. This video described each of the product attributes, which ensures all participants

understand what alternatives they are deciding between in the tasks.

Survey respondents were randomly split into three groups with each group receiving a

different amount of information about the products in the market. A control group engaged in

standard CBC, the first treatment group was shown the prices of products in the market, and a

second treatment group was shown the price of products in relation to multiple attributes of the

product.

The first treatment group was given the price of products in the market in a one-

dimensional bar graph. Price was displayed in ascending order for ease of understanding. Brand

and product details were displayed along each product. Survey respondents placed in this group

were shown a video explaining how to interpret this data prior to making decisions in the choice

tasks. Figure 2 displays the price graph shown to survey participants in this group.

The second treatment group was given the price of products in the market compared to

different attributes of the product in a two-dimensional chart. Price was compared against

number of treatments, time per treatments, time to results, and percent peroxide. Price is

displayed on the x-axis and the attribute value is displayed on the y-axis. Survey respondents

placed in this group were also shown a brief video explaining how to interpret this data. Figure 3

displays one of four graphs shown to survey participants in this group.

The survey was distributed across the United States to over 1,000 participants even across

age, ethnicity, gender, and census region categories. The survey was designed to take allow

participants to complete in 10 minutes or less. Survey participants were 18 or older. The survey

was offered on PCs, laptops, tablets, and mobile phones.

A second survey was added to this research to examine the differences between qualified

and non-qualified individuals. An non-qualified survey participant was an individuals screened

out on one or more of the following criteria:

1. He or she did not make hygienic purchases for his or her household.

2. He or she could not use teeth whitening products for dental reasons.

3. He or she was not interested in purchasing teeth whitening products.

We remove these individuals in CBC studies because they are not qualified to make purchasing

decisions for the product we are studying.

This second survey allowed non-qualified individuals to finish taking the study when

they would have previously been stopped before answering the choice task questions. The data

from this survey allowed us to track who would have been screened out, had it been a normal

CBC study. This data was collected through Amazon Mechanical Turk (MTurk) and was

collected on approximately 2,000 participants.

Data Analysis and Results

Initial model-free results showed there were no significant differences between the

proportion of participants selecting the no-choice option among the three experimental groups

(Figure 4).

We fit a hierarchical multinomial logit model to the data. This model allows for

respondent heterogeneity, meaning that model takes into account differences between survey

respondents. The posterior distribution of the model was compared across experimental groups

for all attributes studied. Comparing the first treatment group to the control group, there were

three attributes in which the differences across treatment conditions were significant because the

upper level coefficients were different from zero. The respondents in the first treatment group

valued products with those attributes significantly more than products without those attributes.

Both pen and trays + gel levels of the form attribute and 25 minutes of the time per treatment

attribute showed statistical significance. Figure 5 shows the posterior distribution for attribute 7

(pen) between the control and first treatment groups.

We then looked into the participants that would have normally been screened out of our

survey. These participants are referred to as non-qualified because they do not make hygienic

purchases for their household, cannot use teeth whitening products, or are not interested in

purchasing teeth whitening products. Given more information about the market, non-qualified

survey respondents were less likely to select the no-choice option (Figure 6).

This suggests that individuals who are informed and interested in the market do not

change their perspective of the no-choice option given more information, as shown in the initial

descriptive results. Given respondents who were qualified properly, the improvement in a CBC

study may not be significant.

Discussion

The results from fitting the model to our data show statistically significant differences in

the upper level coefficients for three variables in the first treatment group. Participants in this

group value products more given they have these attributes. The lack of differences in the

descriptive analysis using SSI data suggests that CBC is a robust method to value attributes of a

product. While participants placed in the control group were given no market information, the

results of the study showed little differences of outcomes between all three groups. Analyzing

the results from non-qualified survey respondents shows the additional price data informs

consumers about the market and gives them a lower propensity to select the no-choice option.

Overall, choice-based conjoint analysis is a robust tool for the marketing research field.

Using appropriate screening techniques gives us participants who already understand the market

and are qualified to make purchasing decisions.

References

Allenby, G., Arora, N., Ginter, J. (1995). Incorporating prior knowledge into the analysis of

conjoint studies. Journal of Marketing Research, 32(2), 152-162.

Brazell, J., Diener, C., Karniouchina, E., Moore, W., Séverin, V., & Uldry, P. (2006). The no-choice option and dual response choice designs. Marketing Letters, 17(4), 255-268.

Caldwell, L. (2015). Making conjoint behavioural. International Journal of Market Research, 57(3), 495-501.

Hsee, C. K., Zhang, J. (2010). General Evaluability Theory. Association for Psychological Science, 5(4), 343-355.

Appendix

Figure 1. One of twelve choice tasks used in the CBC study.

Figure 2. Graph shown to participants in the first treatment group.

Figure 3. One of four graphs shown to participants to participants in the second treatment group.

Figure 4. Initial descriptive results for participants’ propensity to select the no-choice option across experimental groups.

WhiteningSystem$11.87,14

3DWhitestripsGentleRoutine$27.79,14

WhiteningPen$20.09,14

2HourWhiteningKit$22.99,1

3DProfessionalEffectsWhitestrips$43.99,20

WhiteningKit$59.99,8.5

ProLightDentalWhiteningSystem$34.05,15

2MinuteWhitePen$12.99,12

WhiteningPen$28.99,13

0

5

10

15

20

25

$- $10.00 $20.00 $30.00 $40.00 $50.00 $60.00 $70.00

AverageNum

bero

fTreatmen

ts

Price

Pricevs.NumberofTreatments

20.690% 20.200% 19.995%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

ProportionofParticipantsSelectingtheNo-ChoiceOption

ExpCond1

ExpCond2

ExpCond3

Figure 5. Statistical significance in δ7,2.

Figure 6. Non-qualified respondents’ propensity to select the no-choice option.

23.989%

24.720%

23.991%

20.873%

25.021%

23.445%

18.000%

19.000%

20.000%

21.000%

22.000%

23.000%

24.000%

25.000%

26.000%

Qualified Non-Qualified

PropensitytoSelecttheNo-ChoiceOptionusingMTurkResults

ExpCond1

ExpCond2

ExpCond3