Anomalies of a Theory of Rationality. How Paradoxes Inspired Progress in

Decision TheoryAndreas Diekmann

ETH Zurich

Leopoldina Symposium, Halle, September 23rd, 2018

A visual paradox. (Michael Phillips/Getty Images

A journey from the early days of the emergence ofprobability & statistics to modern decision theory.

Probability, utility theory, rational decision making are verymuch related and build on each other.

A word about (modern) terminology to avoidmisunderstanding. There are three types of decisionsituations:

► Decision-making under certainty: consequences of actions are known with certainty.► Decision-making under risk: Probabilities of consequences are known (e.g. weather forecast 70 % rain, 30% sunny – shall I get an umbrella with me?)► Decision-making under uncertainty: Probabilities of consequences are unknown.

► Thinking about paradoxes (or anomalies) gave riseto solutions. Again, critical consideration led tofurther paradoxes and to new solutions and so on.

► This can be seen as kind of a dialectic process, a spiralof development which seems to be a never endingprocess that brought about still controversial results.

Spiral of Decision Theory

1.Maritime insurance, 14/15th cent.

2. Emergence of probability, Pascal 1654

3. St. Petersburgh paradox 1713

4. Utility theory, Bernoulli 1738

6. Axiomatic foundation of utility andrational decision makingVon Neumann/Morgenstern 1947

8.„Anomalies“ of NM-theory, e.g. ELLSBERG-PARADOX, 1961

10. Psychological decision theories,Kahneman/Tversky and many other theories

https://www.youtube.com/watch?v=DTCnbOwomGg

9. Rational decisionsin strategic games, Harsany/Selten 1988

7. Rational decision makingwith subjective probabilities,Savage 1954

5. Bayes 1763

„Early Examples of Marine Insurance“Florence Edler de Roover, 1945

Bernardo Cambi, a Florentine merchant, gives insight into the insurance business of the second half of the 15th century.

Lancia Baleniera, (Yasuo Azuma)

Venetian GalleyMuseo Storico Navale, Venice

Insurance premium range 1.0 – 14.5 per cent of freight.

► 1.0 per cent: Florentine galley, Porto Pisanoto Naples, November 6, 1472

► 14.5 per cent: Baleniera Duke of Burgundy, Barcelona to Bruges, April 15, 1454

http://www.theropetokyo.org/%E5%B1%95%E7%A4%BA%E4%BC%9A%E4%BD%9C%E5%93%81%E9%9B%86/%E7%AC%AC40%E5%9B%9E%E5%B1%95-2015%E5%B9%B4/%E7%AC%AC40%E5%9B%9E%E5%B1%95-2-8/#40-07

Estimates of Risk Factors by OLS RegressionPremium Y = b0 + b1 VenFlor Galley + b2 other Galley + b3 winter + b4 year + εReference category: other types of ships, N = 55, average premium 7.2 %.

b0 = 9.8* (other boats, e.g. Balienera, carrack, nef etc.)b1 = -6.4* (Venetian/Florentine galley, ► premium 3.4 %)b2 = -4.7* (other galleys, premium 5.1 %)b3 = 2.8 („winter“, begin of voyage November to April, not significant!)b4 = -0.10 (year of voyage, not significant)

► Regression analysis reveals the implicit premium ► But how did the merchant decide about how large the premium

should be?

Estimates of Risk Factors by OLS RegressionPremium Y = b0 + b1 VenFlor Galley + b2 other Galley + b3 winter + b4 year + εReference category: other types of ships, N = 55, average premium 7.2 %.b0 = 9.8* (other boats, e.g. Balienera, carrack, nef etc.)b1 = -6.4* (Venetian/Florentine galley, ► premium 3.4 %)b2 = -4.7* (other galleys, premium 5.1 %)b3 = 2.8 („winter“, begin of voyage November to April, not significant!)b4 = -0.10 (year of voyage, not significant)►Estimate of implicit subjective beliefs of risk factors. But did the subjective beliefs correspond to objective risks? Note e.g. that „winter“ is not significant!►Why did probability and decision theory not arise in the Italian merchant cities? Interestingly: Economic reasons (”its the economy, stupid!”) did not lead to the emergence of probability!►Mathematics of accounting was very well elaborated (Fibonacci’s “liber abacus”) but there was no theory of probability. Actuarial science and probability theoryhave been developed two centuries later!

„Birth“ of Probability (and rational decision making under risk) in 1654

(a well known story to experts of probability theory and some gamblers)• Chevalier de Méré, a passionate gambler, posed two problems. One

of the problems: How should two players share the prize if a game is finished preliminary before the predetermined end?

• For example, both players toss a coin alternatively. A player wins if he/she is first to toss 10 heads or tails. The game stops after a sequence of 8 heads and 9 tails. Is the fair distribution 9:8 or 2:1 or some other distribution?

• de Méré asked Pascal for advice and Pascal, in correspondence with Fermat, solved the problem in a letter of 1654 – Leopoldina was two years old!

Emergence of Probability► Pascal‘s solution: Both players should get the Expected (monetary) value of the game which is: Expected value = Winner‘s prize x Probability of winning; (E = V ∙ P)► In the example, player 1 won 8 games and has to win two more. The probability of winning is ½ x ½ = ¼. (9. Tail and 10. Tail) Player 2 already won 9 games and has to win one more – either in the next round or in the round after the next. The probability is ½ + ¼ = ¾. (9. Head or 9. Tail, 10. Head). Therefore, a fair distribution is ¾ versus ¼ or 3:1.► Idle aristocrat’s passions of gambling stimulated the development of probability theory!Ian Hackings, The Emergence of Probability, 1975Lorraine Daston, Classical Probability in the Enlightenment, 1995

The St. Petersburgh ParadoxNicolas Bernoulli (1713)

A problem solved, a paradox arised!

► Consider the following game: A coin is tossed several times until „tails“ appear. If tails appear in the first round you win 1 ruble. If tails appear in the second round you win 2 rubles. 4 in round three, then 8, 16 and so on.

► How much would you pay for playing this game?

The St. Petersburgh ParadoxNicolas Bernoulli (1713)

► Pascal‘s concept of a fair game: probability of winning x prize Expected value 1st round = ½ x 1 ruble = ½ Expected value 2nd round = ¼ x 2 ruble = ½ Expected value 3rd round = 1/8 x 4 ruble = ½ Expected value 4th round = 1/16 x 8 ruble = ½ And so on …► Expected (monetary) value of the game = ½ + ½ + ½ … is infinite!► A fair bet means that you have to pay an infinite amount of money!► There are several solutions to this problem. The theoretically relevant solution was suggested by Daniel Bernoulli, cousin of Nicholas (1738).

Emergence of utility theory► Daniel Bernoulli suggested that utility not money is of interest.► Utility of money decreases with wealth (decreasing marginal utility).► Bernoulli already chose a logarithmic function for the relation between utility and money.

► Petersburgh paradox has a finite expected value when substituting money for utility = log (money).► Bernoullis solution (although there are some problems with the “solution”) gave rise to utility, marginal utility theory and psycho-physical functions.► However, there was no proper device to measure (cardinal) utility.►This problem was solved by John von Neumann and Oskar Morgenstern in the 1947 edition of the Theory of Games and Economic Behavior.

Utility and Rational Decision Theory(Rational Choice)

► Von Neumann & Morgenstern: Preferences should satisfy a set of axioms. If the axioms are satisfied there is a utility function representing the preferences. (EU = Expected utility theory)► It is also possible to measure utility (like temperature with a thermometer!)► The theory was extended by Savage with subjective utilities (SEU = subjective expected utilities). ► Key axiom (1) transitivity: vanilla icecream (A) is preferred to strawberry (B) and strawberry is preferred to walnut (C).

Then vanilla (A) should be preferred to walnut (C) – otherwise preferences are not consistent, there would be no NM-utility function and the decision maker’s choices are irrational.

Utility and Rational Decision Theory(Rational Choice)

Key axiom 2: Independence of irrelevant alternatives (IIA)Vanilla icecream (A) is preferred to strawberry (B) and strawberry is preferred to walnut (C). Walnut is sold out. A rational customer will still stick to vanilla!

Obviously, people will follow the IIA principle. Do they really?

Vanilla Strawberry Walnut

Customer preferences

1 2 3

What is a Rational Decision?

Definition of Rationality: Decisions of an actor whichsatisfy a set of axioms, particularly:

(i) Axiom of transitivity(ii) Axiom of independence of irrelevant alternatives

Subjective probabilities and strategic games

• Von Neumann & Morgenstern: Expected utility, EU-theory, given „objective“ probabilities.

• Savage, 1954, axiomatization of subjective expected utility, SEU theory

• Axiomatic theory of rational decision making in strategicgames (game theory), Harsanyi & Selten 1988. (Unified normative theory of rational decision making)

Paterson, I., A. Diekmann, 1988. A Paradox in Decision Theory and SomeExperimental Results. Theory and Decision 25: 107-116.

69 of 82 subjects chose A. 42 of the 69 subjects with an A-choicerefused to revise the decision for a 2-point bonus.

But what about this little dice problem?A or B? Your choice! Payoffs depend on the outcome of a die roll.

Anomalies of von Neumann-Morgenstern (NM) Rationality Theory

Anomaly means that subjects’ behavior systematically contradicts the NM and Savage axioms of rationality

►Allais paradox►Zeckhauser’s paradox (A version of Russian roulette)►Ellsberg paradox►Some further anomalies relevant for decision makers:

Examples of cognitive biases

Müller-Lyer illusion (adapted from Warnders 2014)

Source: Binmore 2009: 36

X > Y ?

► Example is not unrealistic: How much is a cancer patient willingto pay for an extremely expensive therapy?

How much are you willing to payfor removing the last bullet? (X)

How much are you willing to payfor removing one of four bullets? (Y)

Situation 1You have to pay X

Situation 2You have to pay Y

Are you willingto pay more insituation 1 thanin situation 2?

Zeckhauser‘s Paradox

Ellsberg Paradox30 Balls 60 Balls

Decision 1 red black or greenA 100 $ 0 $ 0 $

B 0 $ 100 $ 0 $

30 Balls 60 BallsDecision 2 red black or greenC 100 $ 0 $ 100 $D 0 $ 100 $ 100 $

A is preferred to B

D is preferred to C

► The urn contains 90 balls. 30 are red. 60 are black or green.

N = 104 C D

A 7 (7%) 90 (87%)

B 3 (3%) 4 (4%)

►Choice of A and D (or B and C) is inconsistent and violates the axioms of rationality.

Ellsberg-Paradox

104 participants of lecturegame theory, ETH Zurich(autumn semester 2016)

30 Balls 60 BallsDecision 1 red black or greenA 100 $ 0 $ 0 $

B 0 $ 100 $ 0 $

30 Balls 60 BallsDecision 2 red black or greenC 100 $ 0 $ 100 $D 0 $ 100 $ 100 $

N = 104

C

D

A

7 (7%)

90 (87%)

B

3 (3%)

4 (4%)

Ellsberg Paradox30 Balls 60 Balls

Decision 1 red black or greenA 100 $ 0 $ 0 $

B 0 $ 100 $ 0 $

30 Balls 60 BallsDecision 2 red black or greenC 100 $ 0 $ 100 $D 0 $ 100 $ 100 $

A is preferred to B

D is preferred to C

Cancel irrelevant alternatives. Consistent choice would be A and C (or B and D).However, A and D is inconsistent and irrational according to classic utility theory

► Most people violate the key axiom of Independence of irrelevant alternatives (IIA)!

Ellsberg‘s icecream parlor

► You prefer yanilla icecream (A) to strawberry (B) and strawberry to walnut (C). You enter an icecream parlor and a surreal dialogue begins:

• You: “I would like to have vanilla icecream!”• The shopkeeper: “Sorry Sir! But walnut is out!”• You: “Well! Good that you tell me. In this case I want strawberry”!

► This is a violation of the principle of independence of irrelevant alternatives – same as with the Ellsberg paradox!

Ellsberg Paradox30 Balls 60 Balls

Decision 1 red black greenA 100 $ 0 $ 0 $

B 0 $ 100 $ 0 $

30 Balls 60 BallsDecision 2 red black greenC 100 $ 0 $ 100 $D 0 $ 100 $ 100 $

A is preferred to B

D is preferred to C

Why do most people opt for A and D?

► Decision under risk: A is a risky decision with a 33% chance of winning 100 $. D is a risky decision with a 2/3 chance of winning 100 $.► Decision under uncertainty: In contrast, the outcome of B and C is uncertain. ► Subjects pay a price to avoid uncertainty. This is called Ambiguity Aversion (AA)!

Ellsberg Paradox

• Ellsberg inspired hundreds of papers:

1. Experimental studies of the paradox and possible applications.2. Measurement of ambiguity aversion3. Decision theories taking account of the paradox4. Applications in finance, medical decision-making, law and other

areas

Are these paradoxes of academic interest only? –Maybe for some decision theory freaks only?

Three examples of applications:

1. Financial investments► Why do German investors prefer stocks from German companies?► Psychological motive of ambiguity reduction. Investors pay a premium for ambiguity aversion.

2. Law and criminal courts (US plea bargaining system)Segall & Stein, 2006. Ambiguity aversion and the criminal process.► Prosecutor has repeated cases. She or he is not ambiguity averse and relies on expected values. „Successfull“ cases promote the prosecutor‘s career.► The defendent faces a single case. He cannot rely on statistical evidence. She or he is ambiguity averse and has to decide for the less risky alternative.► Hence, there is asymmetric ambiguity aversion. This is exploited by the prosecutor in plea bargaining. The prosecutor will offer harsh plea bargaining alternatives, the defendant’s propensity to accept is increased even if innocent!►There is some evidence confirming this explanation.

3. Evidence Based MedicineBerger, Bleichrodt, Eeckhoudt 2013, Treatment decisions under Ambiguity

• Evidence based medicine relies on expected values but decision makers are often ambiguity averse. This leads to:

• Lower inclination of vaccination when negative side-effects are uncertain.

• Lower tendency of treatment when detrimental effects/mortality rates of e.g. surgery are uncertain.

Green energydefault has an extremely large effect of about80% increase in green energyconsumption (and a correspondingdecrease ofconventionalenergy)

Intervention year 2

Households Businesses

Liebe, Ulf, 2018. Green Energy Defaults Have Massive and Persistent Effects in the Householdand Business Sector; Liebe, Gewinner, Diekmann 2018, funded by NFP71.

10,659 households and 1,139 businesses

Electrical power company changed customer‘s default option► Default before: conventional („grey“) energy. ► Default after: slightly more expensive „green“ energy

ca. 80% ca. 70%

Cognitive Biases: The Default Effect

Cognitive Bias: Base Rate FallacyConsider a HIV test. Assumptions (hypothetical):

► (1) the test is 100 % certain in detecting HIV.(0 % false negative rate. If a test is negative a person has no HIV with certainty.)► (2) the test is 99% certain that a person who has not HIV will get the test result “no HIV”. (1 % false positive rate)► (3) there is a 1 per mill (1 in 1000) proportion of HIV positive people in the population.► A person receives a positive test result. How likely is it that he really has HIV? Intuitive guess? Many lay people and experts (medical doctors, judges etc.) guess that the result is 99 % certain!►one can compute the true figure via Bayes formula but there is a simple alternative …

1000‘000

1000 HIV (1 per mill) 999‘000 No HIV

9990 false positive(1 %)

1000 HIV (0 % falsenegative)

10990 receive a positive test result;Among them 9990 who have no HIV= 91 % false alarm!

Test properties0 % false negative1 % false positive1 % HIV in population

An antidote against cognitive biases. Correctreasoning when using frequencies

► Despite a test quality of 0 % false negative and 1 % false positive: 91 % will receive a wrong test result and will be falsely alarmed!► The answer to the question of how certain the positive HIV test result is 9 %! ► How to mitigate this type of cognitive bias?► Gigerenzer’s psychological decision theory claims that many probability anomalies and fallacies disappear when decision problems are framed in terms of frequencies instead of probabilities!

Prospect Theory (Kahneman & Tversky)► Several anomalies can be explained by prospect theory.► Prospect theory is one of the most systematic and elegant behavioural decision theories …► … although there are still problems not solved by the theory

concave: risk averse for gains

convex: risk loving for loss

Daniel Kahneman & Amos Tversky, 1979: Prospect Theory: An Analysis of Decision under Risk , Econometrica, 47(2): 263-292.

Explaining anomalies by more general theories

► Daniel Ellsberg started hiscareer as a scientist and a coldwar strategist at the RAND corporation. ► RAND is a military think tank on decision making, gametheory, and strategic thinking. 30 Noble prize winners wereaffiliated with RAND. ► However, Ellsberg became a whistle blower, a politicalactivist and a political writer.

„Verleihung des Dresdner Internationalen Friedenspreises an Daniel Ellsberg, 21. Februar 2016“, Wikipedia

The other Ellsberg paradox!

Daniel Ellsberg, „A Memoir of Vietnam and the Pentagon Papers“ (2002)

„On the evening of October 1, 1969, I walked out past the guards‘ desk at the Rand Corporation in Santa Monica, carrying a biefcasefilled with top secret documents, which I planned to photocopy thatnight. The documents were part of a 7000-page top secret study ofU.S. decision making in Vietnam, later known as the Pentagon Papers.“ „I saw it [the Vietnam war] first as a problem, next as a stalemate,

then as a moral and political disaster, a crime.“ „In 1971 I also gave copies to the New York Times, to the Washington

Post, (…), all of which defied the government in printing them for the public to read.“ Richard Nixon wants him behind bars. „Eventually I faced twelve

federals felony charges totaling a possible 115 years in prison (…)“.

Steven Spielberg made a movie aboutthe Pentagon Papers („The Post“, Die Verlegerin“). However, in the Hollywood movie the main characterwas the editor of the Washington Post, Kathrine Graham (Maryl Streep), instead of Daniel Ellsberg.

https://en.wikipedia.org/w/index.php?curid=55743990

More than 3000‘000 Vietnamese and US people died in the Vietnam war. The war was error, deception, a complete disaster. It was clearly a war crime of the US government. Even the former secretary of defense, Robert McNamara, later admitted that the war was a fatal error!USA Today: „Daniel Elsberg risked everything to help end the Vietnam war (…)“He received many awards among them the „Right Livelihood Award“ (2006): "...for putting peace and truth first, at considerable personal risk, and dedicating his life to inspiring others to follow his example.“

Ellsberg himself was not risk averse! He proved to be a courageous risk taker by reason of conscience

Edward Snowden declared that he was inspired by Daniel Ellsberg.

The End

Appendix

Ellsberg Paradox30 Balls 60 Balls

Decision 1 red black or greenA 100 $ 0 $ 0 $

B 0 $ 100 $ 0 $

30 Balls 60 BallsDecision 2 red black or greenC 100 $ 0 $ 100 $D 0 $ 100 $ 100 $

A is preferred to B

D is preferred to C

► Choosing A means that you believe that there are less than 30 black balls in the urn. Then you should choose C because there must be more than 60 red and green balls!► Choosing D means you believe that there are more than 30 black balls in the urn (ignoring indifference). This is an obvious contradiction!

u(L) = 1 (Leben); u(D) = 0 (Tod), zwei Skalenpunkte können unter Beachtungvon u(L) > u(D) frei gewählt werden.A = Leben nach Zahlung von X und Entfernung der letzten Patrone,B = Leben nach Zahlung von Y und Entfernung einer von vier Patronen.

1. Die maximale Zahlungsbereitschaft von X und Y bei Indifferenz ist (Binmore 2009):

1 ∙ A = 5/6 ∙ 1 + 1/6 ∙ 0 = 5/63/6 ∙ B = 2/6 ∙ 1 + 4/6 ∙ 0 → B = 4/6

A > B, d.h. bei der gleichen Sache zu unterschiedlichen Preisen (A „kostet“X und für B zahlt man Y) wird A vorgezogen. Dann muss A billiger sein. Es folgt also X < Y.

2. Es ist richtig, dass die Entfernung einer Patrone den Erwartungsnutzen immerum 1/6 erhöht. Der „Nutzen“ ist für jeden Zustand (1 – x/6) ∙ u(L) + x/6 ∙ u(D) = 1 – x/6 mit x der Anzahl Patronen. Wir suchen aber die mit der Nutzenänderungdurch Entfernung einer Patrone korrespondierende Zahlungsbereitschaft, also M = f(Δu).Diese Nutzen-Geld-Funktion muss nicht notwendigerweise linear sein und ist es auch nicht, wenn die Axiome von Neumann/Morgenstern oder Savage zugrunde gelegt werden (wie beider Rechnung oben). Zwischen den beiden Rechnungen 1. und 2. besteht also keinWiderspruch.

(Das Entscheidungsproblem ist nicht völlig abwegig. Ein realistischeres Beispiel wäredie Zahlungsbereitschaft für ein Krebsmedikament!)

���Anomalies of a Theory of Rationality. �How Paradoxes Inspired Progress in �Decision Theory�Slide Number 2Slide Number 3Slide Number 4Spiral of Decision Theory„Early Examples of Marine Insurance“�Florence Edler de Roover, 1945Slide Number 7Estimates of Risk Factors by OLS RegressionEstimates of Risk Factors by OLS Regression„Birth“ of Probability (and rational decision making under risk) in 1654�(a well known story to experts of probability theory and some gamblers)Emergence of ProbabilityThe St. Petersburgh Paradox�Nicolas Bernoulli (1713)�A problem solved, a paradox arised!The St. Petersburgh Paradox�Nicolas Bernoulli (1713)Emergence of utility theoryUtility and Rational Decision Theory� (Rational Choice)Utility and Rational Decision Theory� (Rational Choice)What is a Rational Decision?Subjective probabilities and strategic gamesSlide Number 19Anomalies of von Neumann-Morgenstern (NM) Rationality TheorySlide Number 21Ellsberg ParadoxSlide Number 23Ellsberg ParadoxEllsberg‘s icecream parlorEllsberg ParadoxEllsberg ParadoxAre these paradoxes of academic interest only? – Maybe for some decision theory freaks only?2. Law and criminal courts (US plea bargaining system)3. Evidence Based MedicineGreen energy default has an extremely large effect of about 80% increase in green energy consumption (and a corresponding decrease of conventional energy)Cognitive Bias: Base Rate FallacySlide Number 34An antidote against cognitive biases. Correct reasoning when using frequenciesProspect Theory (Kahneman & Tversky)► Daniel Ellsberg started his career as a scientist and a cold war strategist at the RAND corporation. �► RAND is a military think tank on decision making, game theory, and strategic thinking. 30 Noble prize winners were affiliated with RAND. �► However, Ellsberg became a whistle blower, a political activist and a political writer.Daniel Ellsberg, „A Memoir of Vietnam and the Pentagon Papers“ (2002)Slide Number 39Ellsberg himself was not risk averse! He proved to be a courageous risk taker by reason of conscienceSlide Number 41Slide Number 42AppendixEllsberg ParadoxSlide Number 45