Past research in decision making has shown that when solving certain types of probability estimation...

1
Past research in decision making has shown that when solving certain types of probability estimation problems, groups tend to exacerbate errors commonly found at the individual level (e.g., Tindale, Smith, Thomas, Filkins, & Sheffey, 1996, Tversky & Kahneman, 1974). These errors are due to group members' reliance on shared task representations to reach the final decision for the group. In other words, members of the group share the same idea for how to best solve a given problem. In most cases, these shared task representations lead individuals and groups to the correct answer, but for certain problems this intuitive logic is actually incorrect. Social information processing favors group members who make errors in these problem domains since their incorrect logic seems to make sense to other members of the group. The current study used the judge-advisor system paradigm (e.g., Sniezek & Buckley, 1995) to provide participants with information from others to investigate whether having feedback from others would reduce errors in a Conjunctive Problem: Description 1 Jeff is 32 years old. He is very intelligent and takes most things in life pretty seriously. In college he majored in engineering and mathematics. Jeff is somewhat of a loner and lives by himself in a comfortable condominium. In spaces provided at the right of each statement, please estimate the probability (from 0% = absolutely cannot be true to 100% = absolutely must be true) that each of the statements about Jeff below is true. Then, please rate how confident you are in your probability estimate by circling the number on the scale provided that best represents your level of confidence. The Likely-Unlikely Conjunctive Estimate (LU) Jeff is employed in the information technology field. (Likely advice: 80, 70) Jeff is the social director of his condominium association. (Unlikely advice : 20, 30) Jeff is employed in the information technology field and is the social director of his condominium association. (Likely-Unlikely advice: both correct = 15, 20; both incorrect = 55, 50; one correct and one incorrect = 15, 50) Correct Justification: Since people who are social directors and information technicians are a subset of all social directors, my estimate is somewhat below what I gave for social director. Incorrect Justification: Since it's very likely that Jeff is an information technician, but very unlikely that he is a social director, my estimate was closer to what I said for information technician. Kahneman, D., Slovic, P., & Tversky, A. (Eds.) (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, MA: Cambridge University Press. Sniezek, J. A., & Buckley, T. (1995) Cueing and cognitive conflict in judge-advisor decision making. Organizational Behavior and Human Decision Processes 62, 159–174. Tindale, R. S., Smith, C. M., Thomas, L. S., Filkins, J., & Sheffey, S. (1996). Shared representations and assymetric social influence processes in small groups. In E. Witte, & J. Davis (Eds.) Understanding Group Behavior: Overall, receiving consistent correct advice from others improved performance regardless of whether or not rationales were provided. However, the effect appears mainly due to conformity. Analysis of correctness of logic produced results inconsistent with those for correctness of estimates. Specifically, a correct logic was not shown to increase the likelihood of a correct estimate. Thus, it appears that participants merely repeated the logic suggested by advisors but did not take it into account upon making their own estimates. Taken together, these findings indicate that correct advice can improve performance, but any improvement understanding or learning from the advice provided by others. The conjunctive fallacy is one type of error that individuals tend to make when estimating the probability of two events occurring simultaneously. The error is even greater at the group level. This error is committed when the problem solver believes that the probability of events A and B occurring at the same time is higher than the probability of either event A or B occurring alone. However, the probability of any two events occurring simultaneously must be less than the probability of the event which is the least likely of the two to occur. Thus, the probability of events A and B occurring at the same time must be less than or equal to the probability of the less likely event. The conjunctive fallacy has been previously studied using the “Linda the bank teller” problem. The current study utilized a similar task. INTRODUCTION THE CONJUNCTIVE FALLACY DESIGN AND METHOD ANALYSIS AND RESULTS CONCLUSIONS REFERENCES CONJUNCTIVE PROBABILITY ESTIMATION IN A JUDGE-ADVISOR PARADIGM Rebecca Starkel, Elizabeth Jacobs, Lindsay Nichols, Maura Tobin, & Scott Tindale Loyola University Chicago Chi-square analyses were conducted to compare correctness of estimates against the control group. In comparison with the control, significant differences emerged for various types of conjunctive estimates in various conditions. For LL, 2LU, and 2LL, having 2 correct advisors led to significantly higher percentages of correct estimates among participants. For 2UU, one correct advisor offering rationale led to a significantly higher percentage of correct estimates. For LU and UU, having 2 incorrect advisors led to significantly lower estimates. Additionally, chi-square analyses were conducted to compare correctness of logic against the control group. In comparison with the control significant differences emerged for several types of conjunctive estimates. When participants had 2 incorrect advisors, a significantly lower percentage of participants offered a correct logic for LU. When participants had 2 correct advisors offering rationale, a significantly higher percentage of participants offered a correct logic for LL, and with only one correct advisor offering rationale, a significantly higher RESEARCH QUESTIONS Certain estimations are prone to the conjunctive fallacy and have been shown to be exacerbated at the group level. The current study was designed to determine whether receiving advice for making these types of estimations would improve performance at the individual level. Questions posed included: •Will correct advice improve performance? •Does correct advice need to be consistent in order to be effective? •Will correct advice be enough, or will some type of EXAMPLE MATERIALS Correctness of Estimates (Versus Control, p<.05) Correctness of Logic (Versus Control, p<.05) Design •Judge-Advisor paradigm •2 (information provided: estimate only vs. estimate + justification) X 3 (advisor correctness: both correct vs. both incorrect vs. one correct and one incorrect). •Separate control group which received no information from advisors Method •391 Psychology 101 students at LUC arrived at the computer lab •2 conjunctive problems -students read descriptions of 2 individuals and were asked to estimate the probability of various statements being true for each individual •2 sets of estimates -1 st problem : received information from 2 advisors before making own estimates (with advice) -2 nd problem: received no information from advisors before making own estimates (without advice) Without Rationale 2 incorrect LU 2 incorrect + rationale LL 2 incorrect UU 2 incorrect 2LU 2 correct + rationale 2LL 2 incorrect 2UU Both incorrect (rationale) 2 correct LL 2 incorrect LU 2 incorrect UU 2 correct 2LL Both incorrect (rationale) 1 correct 1incorrect 2UU Both incorrect (rationale) RESULTS CONTINUED... Conjunctive Estimate Abbreviations LU First likely- unlikely, Problem 1 LL Likely-likely UU Unlikely- unlikely LU2 Second likely- unlikely 2LU First likely- unlikely Problem 2 2LL Likely-likely 2UU Unlikely- unlikely 2LU2 Second likely- unlikely Problem 2 Problem 1 Problem 1 Problem 2 2 incorrect UU

Transcript of Past research in decision making has shown that when solving certain types of probability estimation...

Page 1: Past research in decision making has shown that when solving certain types of probability estimation problems, groups tend to exacerbate errors commonly.

Past research in decision making has shown that when solving certain types of

probability estimation problems, groups tend to exacerbate errors commonly

found at the individual level (e.g., Tindale, Smith, Thomas, Filkins, & Sheffey,

1996, Tversky & Kahneman, 1974). These errors are due to group members'

reliance on shared task representations to reach the final decision for the group. In

other words, members of the group share the same idea for how to best solve a

given problem. In most cases, these shared task representations lead individuals

and groups to the correct answer, but for certain problems this intuitive logic is

actually incorrect. Social information processing favors group members who

make errors in these problem domains since their incorrect logic seems to make

sense to other members of the group. The current study used the judge-advisor

system paradigm (e.g., Sniezek & Buckley, 1995) to provide participants with

information from others to investigate whether having feedback from others

would reduce errors in a conjunctive probability task at the individual level.

Conjunctive Problem: Description 1

Jeff is 32 years old. He is very intelligent and takes most things in life pretty seriously. In college he majored in engineering and mathematics.

Jeff is somewhat of a loner and lives by himself in a comfortable condominium.

In spaces provided at the right of each statement, please estimate the probability (from 0% = absolutely cannot be true to 100% = absolutely must

be true) that each of the statements about Jeff below is true. Then, please rate how confident you are in your probability estimate by circling the

number on the scale provided that best represents your level of confidence.

The Likely-Unlikely Conjunctive Estimate (LU)

Jeff is employed in the information technology field.

(Likely advice: 80, 70)

Jeff is the social director of his condominium association.

(Unlikely advice: 20, 30)

Jeff is employed in the information technology field and is the social director of his condominium association.

(Likely-Unlikely advice: both correct = 15, 20; both incorrect = 55, 50; one correct and one incorrect = 15, 50)

Correct Justification: Since people who are social directors and information technicians are a subset of all social directors, my estimate is

somewhat below what I gave for social director.

Incorrect Justification: Since it's very likely that Jeff is an information technician, but very unlikely that he is a social director, my

estimate was closer to what I said for information technician.

Kahneman, D., Slovic, P., & Tversky, A. (Eds.) (1982). Judgment under uncertainty:

Heuristics and biases. Cambridge, MA: Cambridge University Press.

Sniezek, J. A., & Buckley, T. (1995) Cueing and cognitive conflict in judge-advisor

decision making. Organizational Behavior and Human Decision Processes 62,

159–174.

Tindale, R. S., Smith, C. M., Thomas, L. S., Filkins, J., & Sheffey, S. (1996). Shared

representations and assymetric social influence processes in small groups. In E. Witte,

& J. Davis (Eds.) Understanding Group Behavior: Consensual Action by Small

Groups (Vol.1, pp. 81-103). Mahwah, NJ: Lawrence Earlbaum Associates.

Overall, receiving consistent correct advice from others improved performance

regardless of whether or not rationales were provided. However, the effect appears mainly

due to conformity. Analysis of correctness of logic produced results inconsistent with those

for correctness of estimates. Specifically, a correct logic was not shown to increase the

likelihood of a correct estimate. Thus, it appears that participants merely repeated the logic

suggested by advisors but did not take it into account upon making their own estimates.

Taken together, these findings indicate that correct advice can improve performance, but

any improvement seems to be the result of conformity as opposed to understanding or

learning from the advice provided by others.

The conjunctive fallacy is one type of error that individuals tend to make

when estimating the probability of two events occurring simultaneously. The error is

even greater at the group level. This error is committed when the problem solver

believes that the probability of events A and B occurring at the same time is higher

than the probability of either event A or B occurring alone. However, the

probability of any two events occurring simultaneously must be less than the

probability of the event which is the least likely of the two to occur. Thus, the

probability of events A and B occurring at the same time must be less than or equal

to the probability of the less likely event. The conjunctive fallacy has been

previously studied using the “Linda the bank teller” problem. The current study

utilized a similar task.

INTRODUCTION

THE CONJUNCTIVE FALLACY

DESIGN AND METHOD

ANALYSIS AND RESULTS

CONCLUSIONS

REFERENCES

CONJUNCTIVE PROBABILITY ESTIMATION IN A JUDGE-ADVISOR PARADIGM

Rebecca Starkel, Elizabeth Jacobs, Lindsay Nichols, Maura Tobin, & Scott TindaleLoyola University Chicago

Chi-square analyses were conducted to compare correctness of estimates against the control group. In comparison with the control,

significant differences emerged for various types of conjunctive estimates in various conditions. For LL, 2LU, and 2LL, having 2 correct advisors

led to significantly higher percentages of correct estimates among participants. For 2UU, one correct advisor offering rationale led to a

significantly higher percentage of correct estimates. For LU and UU, having 2 incorrect advisors led to significantly lower estimates.

Additionally, chi-square analyses were conducted to compare correctness of logic against the control group. In comparison with the

control significant differences emerged for several types of conjunctive estimates. When participants had 2 incorrect advisors, a significantly

lower percentage of participants offered a correct logic for LU. When participants had 2 correct advisors offering rationale, a significantly higher

percentage of participants offered a correct logic for LL, and with only one correct advisor offering rationale, a significantly higher percentage of

participants offered a correct logic for LU2. When participants had 2 incorrect advisors but no rationale, a significantly higher percentage of

participants offered a correct logic UU, 2LL, and 2UU.

RESEARCH QUESTIONSCertain estimations are prone to the conjunctive fallacy and have been shown to

be exacerbated at the group level. The current study was designed to determine

whether receiving advice for making these types of estimations would improve

performance at the individual level. Questions posed included:

•Will correct advice improve performance?

•Does correct advice need to be consistent in order to be effective?

•Will correct advice be enough, or will some type of rationalization or

justification be needed?

EXAMPLE MATERIALS

Correctness of Estimates (Versus Control, p<.05)

Correctness of Logic (Versus Control, p<.05)

Design

•Judge-Advisor paradigm

•2 (information provided: estimate only vs. estimate + justification) X 3 (advisor correctness: both correct vs. both incorrect

vs. one correct and one incorrect).

•Separate control group which received no information from advisors

Method

•391 Psychology 101 students at LUC arrived at the computer lab

•2 conjunctive problems

-students read descriptions of 2 individuals and were asked to estimate the probability of various statements being true for each individual

•2 sets of estimates

-1st problem : received information from 2 advisors before making own estimates (with advice)

-2nd problem: received no information from advisors before making own estimates (without advice)

Without Rationale

2 incorrect LU

2 incorrect + rationale

LL

2 incorrect UU

2 incorrect 2LU

2 correct + rationale

2LL

2 incorrect 2UU

Both incorrect (rationale)

2 correct LL

2 incorrect LU

2 incorrect UU

2 correct 2LL

Both incorrect (rationale)

1 correct 1incorrect

2UU

Both incorrect (rationale)

RESULTS CONTINUED...

Conjunctive Estimate Abbreviations

LU First likely-unlikely,

Problem 1LL Likely-likely

UU Unlikely-unlikely

LU2 Second likely-unlikely

2LU First likely-unlikely

Problem 22LL Likely-likely

2UU Unlikely-unlikely

2LU2 Second likely-unlikely

Problem 2Problem 1

Problem 1 Problem 2

2 incorrect UU