What do we think we know when we don’t

know much?

Helen Pushkarskaya, Department of Agricultural Economics, University of Kentucky

Sharon Alvarez,Fisher College of Business, The Ohio State University

What is this study about? Theoretically there are three kinds of

decision making environments with less than perfect information about the outcome of a decision: Risk Ambiguity Uncertainty (Sample Space Ignorance)

Risk Both the possible outcomes, and the

probability of those outcomes are known before a decision is made

example: roll balanced dice (6 outcomes, each with probabilities 1/6)

Ambiguity The possible outcomes of a decision are

known before the decision is made, but the probability of those different outcomes occurring are not known, when a decision is being made

example: roll unbalanced dice (6 outcomes, but probabilities are unknown)

Uncertainty (sample space ignorance)

Neither the possible outcomes nor the probabilities of those outcomes are known when a decision is being made

example: roll a dice with unknown number of sides (not all outcomes are known, so it is not possible to define the probabilities)

What is this study about? Are there any practical reasons to distinguish

between these three environments?

In other words,

when decision-makers are in different environments do they recognize it?

Do they act differently in risk, ambiguity and uncertainty?

Risk vs. Ambiguity

Several experimental studies demonstrated that people do distinguish between risk and ambiguity (for survey see Camerer & Weber, 1992)

Risk vs. Ambiguity Decision-makers don’t just assign

subjective probabilities in ambiguous environments and act as if they were in risky environments

e.g. many demonstrate ambiguity aversion

Ambiguity vs. Uncertainty

Do they distinguish between ambiguity and uncertainty?

Our study starts to investigate this question, we use expert bias as an example

Research question

How does expert bias affect decision makers in uncertain environments? Is it in the same way as in ambiguous environments or in a different way?

Ambiguity and Expert bias: Results of Fox and Tversky (1995) suggest that if

people feel that they have more expertise about the environment, then they are more assured about their estimate of the probabilities.

i.e. experts tend to perceive that more information is available than actually is in ambiguous environments

Uncertainty and Expert bias

Results of Fischhoff, Slovic, and Lichtenstein (1978) suggest that experts are less sensitive to the representation bias

Uncertainty and expert bias A possible implication of this finding is that

experts are more likely than non experts to accept that some information about the possible outcomes is missing

Is that so?

Testable hypothesis In uncertain environments experts are more

likely than non experts to admit that not all the future outcomes are known before the decision is made

i.e. something unexpected can happen.

Method

Three scenarios of different uncertain environments (business, computer game, and life events) were presented to 4 different subject groups with different level of expertise in the business environment (naïve, mere exposure, experience/ expertise to the business environment).

Method

Respondents were asked to evaluate whether each scenario is Risky (known outcomes and probabilities), Ambiguous (known outcomes, but unknown probabilities), or Uncertain (unknown outcomes and probabilities)

Survey instrument

In all of the following studies the respondents were given one of the three following scenarios.

Scenario 1 (business) You are a successful businessman and you have

decided to expand. You have an opportunity to buy a controlling interest of a nano-technology firm, and have to make a decision by considering all the possible outcomes. This firm holds a patent on a technology that can identify the potential for breast or colon cancer before people are born. This technology can also alter the genetic makeup in these unborn individuals so that they will not get cancer in their 50’s or 60’s. Up until this point, no hospital, doctor, or insurance company is willing to promote this technology. Researchers continue to work on the technology.

Scenario 2 (computer game) You are playing the new game “Halo 4” that continues

the story of the Master Chief, the genetically enhanced super-soldier who is the only human ever to successfully defy the Covenant--a coalition of alien races on a murderous march toward Earth. Their defeat at the ancient Halo artifact was only temporary, and they are pursuing their goal--the complete obliteration of humankind--with renewed zeal. You have survived for the first 10 minutes. You have entered the empty room and are waiting for what will happen next. Have you ever played Halo 1 or Halo 2 games? □ Yes □ No

Scenario 3 (life events)

You and your friend have decided to make the following bet: both of you make a prediction on where each of you will work in the year 2016. Then, you plan to meet in 2016 and check whether your prediction was right. Whoever has made the closest prediction wins the bet.

Survey instrument

Note that these scenarios were intended “to sound” uncertain, i.e. hypothetically in all these scenarios a decision-maker does not know all the potential outcomes that will follow his decision (at least in opinion of the investigators).

Survey instrument

However, the objective in this study was not to find the subject group that gives “the correct answer,” but to investigate how the perception varies among different subject groups with different level of expertise.

Subjects Three subject groups:

Freshmen at the University of Kentucky Seniors in Economics at the University of

Kentucky MBA’s at The Ohio State University Real Entrepreneurs

Subjects and levels of their knowledge about the environments

Business Computer game

Life events

freshmen naïve Experience/

expertise

Some exposure

Seniors in economics

Exposure to general theory

Experience/

Expertise (less)

Some exposure

MBA’s Environment specific exposure

Experience/

Expertise (less)

Weren’t tested

Real entrepreneurs

Experience/ expertise

Weren’t tested Weren’t tested

Procedure

Respondents were given the definitions of Risky, Ambiguous, and Uncertain environments as well as several examples

of each environment

Procedure

The respondents were told that each of them would be given one scenario randomly chosen from the pack of three (in all studies, including studies 3 and 4), and that they would have to decide whether this scenario describes a risky, an ambiguous or an uncertain environment

Procedure

If the respondents decide that the environment is Risky, they would have to list all of the outcomes and the probabilities of those outcomes

Procedure

If the respondents decide that the environment is Ambiguous then they would have to list all the outcomes

Procedure

If the respondents decide that the environment is Uncertain, they would have to list any number of possible outcomes and estimate the probability that something unexpected will happen

ProcedureThis procedure was chosen to avoid two potential problems:

First, eliminate the potential effect of the comparison among different scenarios;

Second, to minimize the possibility that respondents would try to simply “guess the right answer.”

Study (pilot tests)

Only pilot tests were completed at this point (~10 subjects per group per scenario), therefore all the further results and conclusions are preliminary.

Study 1 We survey 30 undergraduate first year students at

the University of Kentucky with different levels of knowledge about three different scenarios.

This group has high level of expertise in computer game environments, medium (hypothetically exposure and some experience) in life events, and no experience and no exposure (naïve) in business environments.

Results of study 1Scenario 1 (business)

Scenario 2 (computer game)

Scenario 3

(lifetime events)

Identified as Risky

20%(2 out of 10)

0 10%(1 out of 10)

Identified as Ambiguous

70%(7 out of 10)

22%(2 out of 9)

50%(5 out of 10)

Identified as Uncertain

10%(1 out of 10)

78%(7 out of 9)

40%(4 out of 10)

Study 1: comments

These results suggest that students perceive that they know less about what can happen in a computer game, which is a familiar environment for freshmen (75% of the students indicated that they have played Halo 1 or Halo 2 games), than about their future (scenario 3), and even less than about completely unfamiliar business opportunity (scenario 2).

Study 2 The same set of scenarios using the same

procedure was offered to 30 seniors in economics, enrolled in a class focused on risk and insurance at the University of Kentucky.

These students had relatively strong background in economics and specifically were trained to analyze different informational environments.

Results of study 2Scenario 1 (business)

Scenario 2 (computer game)

Scenario 3

(lifetime events)

Identified as Risky

10%(1 out of 10)

25%(2 out of 8)

25%(3 out of 12)

Identified as Ambiguous

50%(5 out of 10)

25%(2 out of 8)

25%(3 out of 12)

Identified as Uncertain

40%(4 out of 10)

50%(4 out of 8)

50%(6 out of 12)

Study 2 comments

These results suggest that formal training in economics and specifically in the theory of insurance markets decreases differences in perception of different environments.

Study 3 We offered Scenarios 1 and 2 using the same

procedure to 17 students, enrolled in the MBA program in the Business School at The Ohio State University, who were taking class specifically designed for future entrepreneurs (82% of them never attempted to open their own business).

Hypothetically these students have mere exposure specifically to business environments, but not relevant experience or expertise.

Results of study 3Scenario 1 (business)

Scenario 2 (computer game)

Identified as

Risky 0 11%

(1 out of 9)

Identified as Ambiguous

63%(5 out of 8)

33%(5 out of 9)

Identified as Uncertain

37%(3 out of 8)

56%(4 out of 9)

Study 3 comments These results suggest that mere exposure to

entrepreneurial environments also increases probability that respondents recognize the business environment as uncertain.

However, a majority of students still believed that the environment described in the computer game is uncertain, and minority of students thought that the business scenario described uncertain environment.

Study 4 We presented Scenario 1 to 11 real

entrepreneurs, i.e. people who have opened at least one business and have worked in the business environment for at least 4 years, but not necessarily had a degree in economics or MBA.

Hypothetically, the respondents from this subject group have experience and/or expertise in environments similar to the one offered in Scenario 1, but not a general knowledge of theoretical definitions.

Results of study 4Scenario 1 (business)

Identified as

Risky 0

Identified as Ambiguous

18%(2 out of 11)

Identified as Uncertain

82%(9 out of 11)

Study 4 comments This result suggests that subjects, who had

real life experience and possible expertise in entrepreneurial environments, were likely to admit that they do not know enough about these environments, or more precisely they know enough about them to realize and admit that they don’t know all the potential outcomes, and that something completely unexpected might occur.

Overall results

The following two tables demonstrate how the perception of the same scenario varies among groups with different level of knowledge about the environment

Scenario 1 (business)Freshmen(naïve)

Seniors (mere

exposure to theory)

MBA’s(mere

exposure to business environment)

Entrepreneurs(experience/

expertise)

Risky 20% 10% 0 0

Ambiguous 70% 50% 63% 18%

Uncertain 10% 40% 37% 82%

Scenario 2 (computer game)Freshmen(experience/ expertise)

Seniors (less experience)

MBA’s(less experience)

Risky 0 25% 11%

Ambiguous 22% 25% 33%

Uncertain 78% 50% 56%

Preliminary conclusion 1 Different subject groups on average perceive

some environments differently in a between-subject design (with no possibility of comparison with other environments and with more knowledgeable individuals).

This difference appears to be related to how well the subjects know the environment they have to evaluate.

Preliminary conclusion 2

Subjects with no experience and no exposure to the entrepreneurial environment are less likely to admit that something unexpected might happen in entrepreneurial environment than subjects with exposure to this environment, but no real life experience.

Preliminary conclusion 3

Subjects with real life experience in entrepreneurial environments are more likely to admit that this environment is complicated enough, and not all the future outcomes are known, than subjects with exposure to this environment, but no real life experience.

Summary It is not clear, based on the available data,

whether experience or expertise are responsible for the differences in perception of the same environments by different subject groups;

Whether the experience or expertise in one type of uncertain environments might affect perception of other uncertain environments;

Summary

And whether age, gender, and other factors influence the perception of uncertain environments.

These and other hypotheses have to be tested further.

Summary Overall this data suggests that experts (or

individuals with experience) in uncertain environments are more likely than individuals with no experience or expertise to admit that some information about the environments is missing, while in ambiguous environments experts tend to think that they know more about the environment then they actually do.

Implications

Therefore, ambiguous environments appear to be different from uncertain environments not only theoretically, and this difference needs to be studied further.