Module 2Decision Theory

Module 2 Topics

• Rationality I: Preference Orders

• Rationality II: Empathy and Morality

• Expected Value & Profit

• Marginal Utility

Module 2 Exam Points

Rank-order preferences

Compute expected value

Compute profit

Compute cost-per-benefit

Rationality I:

Preference Orders

The word ‘rational’ can mean many things.

If I say you're rational, I could mean you're smart, or thoughtful, or well balanced, or all these things.

In decision theory we use the term ‘rational’ to mean something very specific.

VOCABULARY

Rationality is a principle by which decisions are made and actions are taken so as to maximize utility (given available resources and constraints on action).

Note: We'll define ‘utility’ below. For now just think of it as a unit of satisfaction. I eat one potato chip, I get a unit of satisfaction from it; I eat another potato chip, I get additional satisfaction (though perhaps a little less).

Note: We’ll be talking about resources and constraints shortly.

A simple version of rationality has us finding the best way to get from Point A to Point B.

In this maze you can see that some paths take you from entry to exit with less effort than others.

You’re acting rationally if you follow the shortest route.

A

B

A more sophisticated view was described by John von Neumann.

Von Neumann saw rationality as a consistent ranking of preferences.

Goal 1

Goal 2

Goal 3

Goal 1 is preferred to Goal 2, which is preferred to Goal 3.

Some notation gives us a way to express this thought.

NOTATION

𝐺𝑜𝑎𝑙 1 ≻ 𝐺𝑜𝑎𝑙 2 ≻ 𝐺𝑜𝑎𝑙 3

or

𝐺𝑜𝑎𝑙 3 ≺ 𝐺𝑜𝑎𝑙 2 ≺ 𝐺𝑜𝑎𝑙 1

Goal 1

Goal 2

Goal 3

Notice that there’s curve in the arrow.

The curve indicates that we mean

‘preferred over’, not ‘greater than’.

You can read the expression

𝐺𝑜𝑎𝑙 3 ≺ 𝐺𝑜𝑎𝑙 2 ≺ 𝐺𝑜𝑎𝑙 1

as “Goal 3 is preferred less thanGoal 2, which is preferred less than Goal 1.”

Goal 1

Goal 2

Goal 3

TECHNICAL NOTE

Goals can reside on the same tier.

Snickers

Pez or Cashews

Personally, I prefer Snickers to both Pez and Cashews, and I’m indifferent between Pezand Cashews.

NOTATION

𝑆𝑛𝑖𝑐𝑘𝑒𝑟𝑠 ≻ 𝑃𝑒𝑧 ~ 𝐶𝑎𝑠ℎ𝑒𝑤𝑠

Snickers

Pez or Cashews

The tilde implies indifference.

We hope to rank our preferences consistently, so that

• If Goal 1 is preferred to Goal 2 and Goal 2 is in turn preferred to Goal 3; then

• Goal 1 is preferred to Goal 3.

Goal 1

Goal 2

Goal 3

In truth, however, our all-too-human preferences often fail to sort properly.

The grass is always greener on the other side!

Side 1

Side 2

Side 3

You buy Sensible Car, which makes you desire Sports Car.

When you buy Sports Car, you can’t haul stuff from the hardware store and realize you need Truck.

Truck, however, once you start driving it, burns a lot of gasoline, so you want Sensible Car.

And so on....

Sensible Car

Sports Car

Truck

Rationality in our sophisticated definition requires that you both 1) sort your goals properly, and 2) act in accordance with those goals.

Sensible Car

Sports Car

Truck

Consider the following problem.

You most want to go to the big show at Arena and also go on Vacation.

Your next preference would be Small show together with Vacation.

Just below that preference is Arena without Vacation.

Downward from there is Vacationwithout any show.

At the bottom of the preference list is Small without Vacation.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

Here’s the catch: The options cost money and your resources are limited.

• Arena costs $300.

• Small costs $200.

• Vacation costs $400.

• Arena and Small occur on the same night.

• You have $600.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

Let’s give you a sequence of options.

First, I offer a choice between Arena and Small. You pick Arenabecause you prefer it over Small.

But if you take Arena, you’ve spent so much money that you can no longer afford Vacation.

On Day 3 it’s too late to change the decision you made on Day 2.

Day 1• I offer a choice of Arena

or Small, but not both.

Day 2• You select Arena

Day 3

• You can’t afford Vacationand it’s too late to buy the

Small + VacationCombo

Notice:

• The ideal option was never attainable (not enough money). It was simply impossible.

• The option one step down was possible, but not after choosing Arena. It turned into a missed opportunity.

• We end up with just Arena. It's a suboptimal option.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

Impossible

Missed Opportunity

Result

VOCABULARY

• The ideal outcome is defined as the best possible outcome, notwithstanding resources and constraints.

• The optimal outcome is the best possible outcome, given resources and constraints.

• A suboptimal outcome is an outcome valued less than the optimal.

• The nightmare outcome is the worst possible outcome.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

Ideal

Optimal

Suboptimal

Nightmare

Suboptimal

Suboptimal

Note:

• ‘Optimality’ is a term commonly used in game theory.

• ‘Ideal’ and ‘Nightmare’ are made up for this class.

• I use the terms ‘Ideal’ and ‘Nightmare’ to sharpen the intuitive distinction between what is possible and what is realistic.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

Ideal

Optimal

Suboptimal

Nightmare

Suboptimal

Suboptimal

Problem:

• In the real world, preferences sometimes (often?) fail to sort properly.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

The Rational Solution:

• You want to go to Arena and go on Vacation, but you add up the costs and realize that is not going to happen.

• Instead, you decide to go with Small and Vacation.

• It’s not an ideal solution, but it's better than the alternatives.

Day 1

• I offer a choice of Arena or Small, but not both.

Day 2• You select Small

Day 3• And then go on

Vacation

You have achieved optimality.

Think ahead! It’s a good lesson for life.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

Impossible

Optimal

Notice: I gave you a set of binary preference relations:

• Arena + Vacation is preferred to Small + Vacation

• Small + Vacation is preferred to Arena

• Arena is preferred to Vacation

• Vacation is preferred to Small

These binary relations can be combined into the total preference ordering on the right.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

VOCABULARY

• A binary preference relation is an ordered pair of preferences.

• A total preference ordering is the (presumably consistent) result of all binary preference relations.

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

Ideal

Optimal

Suboptimal

Nightmare

Suboptimal

Suboptimal

Of course, we could mix up the binary preference relations and get the same total preference ordering:

• Vacation is preferred to Small

• Small + Vacation is preferred to Arena

• Arena + Vacation is preferred to Small + Vacation

• Arena is preferred to Vacation

Arena + Vacation

Small + Vacation

Arena

Vacation

Small

EXAM POINT

On the exam, you will review a set of binary preference relations then find the total preference ordering.

Rationality II:

Empathy and Morality

It's frequently charged, wrongly, that rational choice ignores human empathy.

• The False Claim: If I'm pursuing my own goals, I must be ignoring yours.

This misguided idea has been helped along by advocates of rational choice theory who do in fact ignore empathy.

You and I don't have to make this mistake, as the intellectual founders of rational choice made plenty of room for moral choices.

Truth be told, my interests can easily encompass your interests.

Every day, you see people defy the notion that everyone is selfish all the time.

• Think about good deeds you have done for your friends and family…

• … and they have done for you!

Don’t be confused by the triviality of the term ‘preference’.

To prefer one thing over another is just a way of saying: “Offered the choice between one object and another, this is the one would I choose.”

There's no problem with me saying I prefer World Peace over Snickers, which I prefer over Cashews.

World Peace

Snickers

Cashews

The fact that World Peace is a moral goal and snacks are a personal taste is immaterial.

Given the choice, I prefer WorldPeace.

World Peace

Snickers

Cashews

Important:

• There's no contradiction between rationality and empathy.

• There's no contradiction between rationality and morality.

• There's no contradiction between rationality and charity.

Example:

I give to the Wounded Warrior Project.

Evidently, I prefer sending some of my money to a good cause over spending it on other things.

I can therefore state a rank-order of my own preferences.

Watch the a clip of Martin Luther King’s 1965 speech at the Brown Chapel in Selma.

Can you rank King’s preferences?

Use the worksheet on the next slide to analyze King’s speech on the following slide.

Copy this diagram.

Use it to record MLK’s preference-order from his speech in Selma (starting at 1:30).

________

________

________

________

Go to the next slide and click to start

Required

Clip from Martin Luther King, “Brown Chapel Speech” (1965)

Bottom line: Rationality is about preference-ordering, not selfishness.

Expected Value & Profit

Question: Why is a stamp included in this solicitation? How could the added expense possibly be worthwhile, given the cost of postage?

Is it certain that I'll give? No.

Is it certain that I'll not give? No.

Is there some chance between 0% and 100% that I'll give money to the cause? Yes!

To better understand the situation, we're going to think in terms of expected value.

VOCABULARY

Expected value is the average value that would accrue from an uncertain event if that event were to occur an infinite number of times.

The concept is a fiction – we can’t actually make the same bet over and over forever – but it's a useful fiction that tells you how much you can expect to be paid from any single wager.

To find the expected value we multiply a payoff by the probability of receiving the payoff.

I urge you to work through the next section slowly.

Pull out a piece of paper and a pencil, and maybe a calculator, to follow along.

Go back and forth on the presentation if the ideas don’t make sense at first.

Imagine I offer you a game.

I hand you a fair die and ask you to throw it. I'll give you a dollar for every dot that comes up. If the die shows a three, I’ll give you three dollars. If you throw a six I'll give you six dollars.

What is the expected value of a single throw?

We can find the answer with simple arithmetic.

There are six possible outcomes, and these outcomes can be represented in a set ′1′, ′2′, ′3′, ′4′, ′5′, ′6′ .

Notice that I’ve used curly brackets. They tell you that the order does not matter at this stage of our analysis.

• I could just as sensibly have written ′5′,′ 3′,′ 2′,′ 4′,′ 1′, ′6′ , but counting up from ‘1’ gives us an easier way to remember what’s going on.

Also notice that I put quotes around the numerals. That’s because these numerals are the names of the outcomes, not their values.

Let’s add values to our outcomes.

To make things simple, we are going to turn our set of outcomes into a tuple.

VOCABULARY

A tuple is an ordered list. That is, it represents a collection of elements indexed by a sequence that must not be changed.

NOTATION

A tuple is commonly denoted with angle brackets. Example: 𝑎, 𝑏, 𝑐 . On the exams we will be using parentheses because Blackboard does not allows angle brackets, but there is little harm in this notation as parentheses also commonly used to indicate tuples.

With a set, the order doesn’t matter:

′1′, ′2′, ′3′, ′4′, ′5′, ′6′ = ′5′,′ 3′,′ 2′,′ 4′,′ 1′, ′6′ .

With a tuple, order does matter:

′1′, ′2′, ′3′, ′4′, ′5′, ′6′ ≠ ′5′,′ 3′,′ 2′,′ 4′,′ 1′, ′6′ .

Let’s call our outcome tuple 𝑂 (for ‘Outcome’) and say that

𝑂 = ′1′, ′2′, ′3′, ′4′, ′5′, ′6′ .

And let’s call our value tuple 𝑋 (because that is the convention) and say that

𝑋 = $1, $2, $3, $4, $5, $6 .

𝑂 = ′1′, ′2′, ′3′, ′4′, ′5′, ′6′ .

𝑋 = $1, $2, $3, $4, $5, $6 .

By counting from left to right on both 𝑂and 𝑋 we can see that outcomes relate to values by their position in the tuple.

For example, ′3′ → 3, or in English we can say, “The outcome ’3’ maps to the value 3.”

Let’s hang onto that idea for a minute.

It will help us produce a general formula for the expected value of our game and others.

Here’s what we can say about our game:

• Each side of the die has an equal chance of coming up. That is, each outcome can occur with equal probability.

• If you were to throw the die a million times, you can expect each side to come up (almost exactly) one-sixth of the time.

• Each time a number comes up, you get paid an amount equivalent to the number shown.

Can you work out the expected value of a single throw? Give it a try before you move on.

If you roll the die a million times (and ignore rounding errors):

• The ‘1’ will appear roughly 166,667 times, giving you $166,667;

• The ‘2’ will appear roughly 166,667 times, giving you $333,333;

• The ‘3’ will appear roughly 166,667 times, giving you $500,000;

• The ‘4’ will appear roughly 166,667 times, giving you $666,666;

• The ‘5’ will appear roughly 166,667 times, giving you $833,333; and

• The ‘6’ will appear roughly 166,667 times, giving you $1,000,000.

Add it all up:

$166,667 + $333,333 + $500,000 + $666,666 + $833,333 + $1,000,000

= $3,5000,000.

If you roll the die a million times, you can expect to walk away with $3.5 million.

In other words, the expected value of throwing the die a million times is $3.5 million.

But in our game, I only let you throw the die once.

Next Question: What would you figure is the expected value of a singletoss.

Try to work it out.

If you throw the die a million times and expect to receive $3.5 million dollars, then throwing the die only one time should get you a millionth of that sum, or $3.50:

$3,500,000

1,000,000= $3.50

That’s the core concept of expected value!

The expected value of an action is the average value you would expect to accrue in the long run.

VOCABULARY AND NOTATION

An event like the coin toss can be denoted by a capital letter such as 𝑋.

This event will lead to one of six possible values, which we can denote with lower-case letters in our tuple 𝑋 = 𝑥1, 𝑥2, 𝑥3, 𝑥4, 𝑥5, 𝑥6 .

These values are related to our tuple of outcomes, which we previously called 𝑂 = 𝑜1, 𝑜2, 𝑜3, 𝑜4, 𝑜5, 𝑜6 .

The six possible outcomes are each associated with a probability of occurring, which we will call 𝑃, so 𝑃 = 𝑝, 𝑝2, 𝑝3, 𝑝4, 𝑝5, 𝑝6 .

Let’s think about this for a moment.

There are six possible outcomes, each of which is equally likely.

Therefore, the probability of throwing a ‘4’ is one in six, or 1

6.

The payoff for throwing a ‘4’ is $4.

One way to think about it, the value of a ‘4’ is the payoff from throwing a ‘4’ multiplied by the probability of throwing a ‘4’.

Formally: $4 ×1

6=

$4

6, or about $0.67.

We can now use the notation we just learned.

𝑋 = $1, $2, $3, $4, $5, $6 .

𝑃 =1

6,

1

6,

1

6,

1

6,

1

6,

1

6.

Remember: Expected value is calculated as the average value of a payoff multiplied by the probability of receiving the payoff.

We have six possible outcomes and six possible payoffs.

Your Job: Find the expected value for each outcome and add those expected values together.

We can do the same computation for all possible outcomes.

$1 ×1

6=

$1

6, or about $0.1667

$2 ×1

6=

$2

6, or about $0.3333

$3 ×1

6=

$3

6, or about $0.5000

$4 ×1

6=

$4

6, or about $0.6667

$5 ×1

6=

$5

6, or about $0.8333

$6 ×1

6=

$6

6, or about $1.0000

Add up the values you can expect from each of the possible outcomes and you get $3.50.

Or, you can think about it this way:

$1 ×1

6+ $2 ×

1

6+ $3 ×

1

6+ $4 ×

1

6+ $5 ×

1

6+ $6 ×

1

6= $3.50

Or, this way:

𝑥1 × 𝑝1 + 𝑥2 × 𝑝2 + 𝑥3 × 𝑝3 + 𝑥4 × 𝑥4 + 𝑥5 × 𝑝5 + 𝑥6 × 𝑝6

= ExpVal 𝑋

where ExpVal 𝑋 stands for the expected value of 𝑋.

VOCABULARY AND NOTATION

The expected value of an event 𝑋, which well will denote as ExpVal 𝑋 , is result of the computation

ExpVal 𝑋 = 𝑥1 ∙ 𝑝1 + 𝑥2 ∙ 𝑝2 + ⋯ + 𝑥𝑘 ∙ 𝑝𝑘

where 𝑥 is the payoff associated with an outcome, 𝑝 is the probability of the outcome’s occurrence, and 𝑘 is the last outcome in our list.

The three dots (ellipses) in ‘+ ⋯ +’ tell us that we could plug more terms into the expression. Also, in our game, 𝑘 = 6, but if we were using a deck of cards we would say 𝑘 = 52 and fill in the ellipses accordingly.

EXAM POINT

On the exam, you’ll review a pair of tuples – one for values and the other for probabilities, then find the expected value.

Returning to the fundraising letter....

Let’s imagine:

• Ten people in a hundred are persuaded by the letter and send back $25.00, while the rest return nothing.

What is the expected value of the mailing?

Try to find the answer before going to the next slide.

What we know:

• The probability of receiving a donation in return for a letter is ten

people in a hundred, or 10

100, or

1

10.

• The probability of the organization receiving nothing in return for a

letter is ninety in a hundred, or 90

100, or

9

10.

• The payoff for receiving a donation is $25, and the payoff for receiving nothing is $0.

• Therefore, 𝑂 = ′𝐷𝑜𝑛𝑎𝑡𝑖𝑜𝑛′, ′𝑁𝑜 𝐷𝑜𝑛𝑎𝑡𝑖𝑜𝑛′ , 𝑋 = $25, $0 ,

and 𝑃 =1

10,

9

10.

ExpVal 𝐿𝑒𝑡𝑡𝑒𝑟 = $25 ×1

10+ $0 ×

9

10= $2.50.

But letters are not free.

If we want to figure out what we can achieve with this mailing, we should compute expected profit.

VOCABULARY AND NOTATION

The expected profit of an event is the result of the computation

𝑃𝑟𝑜𝑓𝑖𝑡 𝑋 = 𝐸𝑥𝑝𝑉𝑎𝑙𝑢𝑒 𝑋 − 𝐶𝑜𝑠𝑡 𝑋

where 𝑃𝑟𝑜𝑓𝑖𝑡 𝑋 stands for the expected profit of 𝑋 and 𝐶𝑜𝑠𝑡 𝑋stands for the cost of gaining that expected value.

What we know:

• ExpVal 𝐿𝑒𝑡𝑡𝑒𝑟 = $25 ×1

10+ $0 ×

9

10= $2.50.

• A letter costs $0.75.

Therefore, 𝐸𝑥𝑝𝑃𝑟𝑜𝑓𝑖𝑡 𝐿𝑒𝑡𝑡𝑒𝑟 = $2.50 − $0.75 = $1.75.

Gambling operates on the same logic.

In the next video, watch how the payoffs are calculated.

Also, beware of strategies suggested by those who profit from gambling. (See if you can spot some bad advice.)

Enjoy the look and feel of the 1980s!

Go to the next slide and click to start

Required

Clip “Gambling” (circa 1988)

EXAM POINT

On the exam, you’ll review a pair of tuples – one for values and the other for probabilities – and a value representing the cost. You'll find the expected profit.

Expected profit is straightforward in that you invest some money and hope to get back more than you put in.

Money in. Money Out.

Similar computations can be made for actions.

Money in. Action Out.

A key metric in electoral politics is the cost per marginal vote gained.

VOCABULARY AND NOTATION

The Cost per Benefit of an action 𝐴, denoted Cost 𝐴 is result of the computation

𝐶𝑜𝑠𝑡𝑃𝑒𝑟𝐵𝑒𝑛𝑒𝑓𝑖𝑡 𝑋 =𝐶𝑜𝑠𝑡 𝐴

𝐵𝑒𝑛𝑒𝑓𝑖𝑡 𝐴

where 𝐶𝑜𝑠𝑡 𝑋 is expense incurred for undertaking the action and 𝐵𝑒𝑛𝑒𝑓𝑖𝑡 𝑋 is the gain accrued.

Alternatively, we could calculate Benefit per Cost as

𝐵𝑒𝑛𝑒𝑓𝑖𝑡𝑃𝑒𝑟𝐶𝑜𝑠𝑡 𝑋 =𝐵𝑒𝑛𝑒𝑓𝑖𝑡 𝐴

𝐶𝑜𝑠𝑡 𝐴

Let’s tackle the idea of marginality.

A marginal vote is one you would not have gained but for the effort you put into the campaign.

If your candidate was going to receive 1,000 votes without running any campaign and 1,002 with the campaign you actually ran, then your final tally includes two marginal votes.

Let’s assume that the campaign cost $100.

Because you earned 2 marginal votes for that expense, then your cost

per marginal vote is $100

2, or $50.

Another example:

If a postcard mailing is going to cost $50,000 and we expect a marginal gain of 2,500 votes, then

𝐶𝑜𝑠𝑡𝑃𝑒𝑟𝐵𝑒𝑛𝑒𝑓𝑖𝑡 𝑋 =$50,000

2,500= $20 𝑝𝑒𝑟 𝑚𝑎𝑟𝑔𝑖𝑛𝑎𝑙 𝑣𝑜𝑡𝑒.

Now the easy part.

I offer you two mail plans:

1. I can get you 2,500 marginal votes for $50,000; or

2. I can get you 1,000 votes at $7.50 per marginal vote and 1,500 votes for another $30,000.

Which plan should you choose?

Both plans get you 2,500 votes, so the key is cost:

1. $50,000; or

2. 1,000 × $7.50 + $30,000 = $7,500 + $30,000 = $37,500.

It’s an easy decision, once you do the arithmetic.

EXAM POINT

On the exam, you’ll review numbers representing the a vote-count accumulated without a campaign, with a campaign, and a cost for the campaign, and then find the Cost per Marginal Vote.

Go to the next slide and click to start

Of course, for decisions to matter, they must be enforceable.

Outside Resources

Outside Resources

• KA: Expected Value

• GT101: Rationality and Utility

• Wikipedia: Expected Value