Lecture 5. Basic Probability

David R. Merrell90-786 Intermediate Empirical Methods

For Public Policy and Management

I think that the team that wins game five will win the series...Unless we lose game five.

-- Charles Barkley

Regularity: Empirical Rule

contains 68% of data

contains 95% of data

contains 99.9% of data

X S

X S2

X S3

How to Verify?

Try Monte Carlo simulations Easy to use Minitab Let’s do that!

Terminology

Probability trial: a process giving observations with uncertain values

Repeated probability trials: independently repeated under the same conditions

Outcome: a most basic happening Event: set of outcomes

Assignment of Probabilities

1. Symmetry--Classical

2. Relative Frequency

3. Betting Odds--Subjective

Classical Approach

Elementary outcomes are equally likely Probability is defined to be the

proportion of times that an event can theoretically be expected to occur

Used in standard games of chance We can determine the probability of an

event occurring without any experiments or trials ever taking place

Example 1 - Rolling a die

Experiment: Roll a die Sample space: S = {1, 2, 3, 4, 5,

6} Number of possible outcomes: 6

P(4) = 1/6 P(even) = 3/6 P(number < 3) = 2/6

Example 2 - Flipping a coin

Experiment: Flip 2 coins Sample space: S = {HH, TH, HT,

TT} Number of possible outcomes: 4

P(both heads) = 1/4 P(at least one tail) = 3/4

Example 3 - Drawing a card

Experiment: Draw a card from a deck of 52

Number of possible outcomes: 52 P(ace) = 4/52 P(diamond) = 13/52 P(red and ace) = 2/52

Relative Frequency Approach

Used when classical approach is not applicable and repeated probability trials are possible

Probability is the proportion of times an event is observed to occur in a large number of trials

Example 4--Relative Frequencies

In 1985, 22.9% of whites were below the poverty level

In 1977, the percent urban in Iraq was 64.

In 1984, the divorce rate in Maine was 3.6 per 1000 population. (Problems here!)

Law of Large Numbers

300200100

0.0

-0.5

-1.0

Index

aver

age

“Law of Small Numbers”

Toss a coin 1000 times and it will show up heads 500 times???

“Law of Averages”

“I’ve lost money every time I bought a stock...I’m due!”

Subjective Approach

Used when repeated probability trials are not feasible.

Probability is subjective--an educated guess, a personal assessment

Well-Calibrated Probability Forecaster

Link subjective probability to repeated probability trials

P(MSFT goes up tomorrow) = .55 Does it go up 55% of the time?

Example 5--Subjective Probability

What is the probability that the Pittsburgh Steelers will win next week?

What is the probability that Al Gore will be elected president in the year 2000?

Odds vs. Probabilities

Odds are a restatement of probability If the probability that an event will occur is 3/5, then the odds in

favor of the event occurring are 3:2

Odds against an event occurring are the reverse of odds in favor of occurring. In this case 2:3.

To calculate the probability, given the odds 1:3

1 1

1 + 3 4probability is 1/4

Odds

Odds of a:b in favor of an event A

Bet in Favor Bet Against

A Occurs

A Does Not

b -b

-a a

Probability Notation P(A) - probability that event A occurs P(A’) - probability that event A will not occur

(A’ is the complement of A) P(A B) - probability that A will occur or B

will occur or both (Union of A and B) P(A B) - probability that A and B will occur

simultaneously (Joint probability of A and B) P(A | B) - probability of A, given that B is

known to have occurred. (Conditional probability)

Probability Axioms

1. P(A) > 02. P(S) = 13. Ai mutually exclusive,

P A P Ai i( ) ( )

Addition Law for Probability

P(A or B) = P(A) + P(B) - P(A and B)

Example: A left engine functions B right engine functions

“Proof by Paint”

A B1 1 0

“paint and scrape”

A B1 2 12 1

P A B P A B P A P B P A B( ) ( ) ( ) ( ) ( )or

If Mutually Exclusive ...

P(A or B) = P(A) + P(B)

Note simplification of Addition Rule

If Independent ...

P(A and B) = P(A)P(B)

Note simplification of Multiplication Rule

Some Connections ...

Logic Set Arithmetic Simplification

and x independence

or + mutually exclusive

Note: independence is NOT mutual exclusivity

Multiplication Law for Probability

P(A and B) = P(A B) = P(A)P(B|A) = P(A|B)P(B)

Example

Sell cocaine and go to jail A B

Example 6--Probability Calculations

P(adult male is a Democrat) = 0.6,P(belongs to a labor union) = 0.5P(Democrat and labor union) = 0.35, Find the probability that an adult male

chosen at random: is a Democrat or belongs to a labor union does not belong to a labor union is a Democrat given that he belongs to a

labor union

Conditional Probability

Events A, B

P(A and B) = P(B |A)P(A) = P(A|B)P(B)

Definition:P B A

P A B

P A( | )

( )

( )

Example 7--Conditional Probability

{ , , , }

{ , , }

{ , , , }

1 2 10

3 5 7

2 3 5 7

3

4

A

P

P

odd number = {1, 3, 9}

B = prime number = {2, 3, 5, 7}

P(A|B) =P(A B)

P(B)

Contingency Table

Help determine probabilities when we have two variables

Joint and conditional probabilities are in the cells

Marginal probabilities are on the “margins” of the table

Educational Achievement: Coding of Ordinal Variable

1 if grade 4 or less 2 if grades 5-7 3 if grade 8 4 if high school incomplete (9-11) 5 if high school graduate (12) 6 if technical, trade, or business after high

school 7 if college/ university incomplete 8 if college/university graduate or more

Educational Achievement Table

Education Female Male Total

No. % No. % No. %

3 1 0.21% 1 0.21% 2 0.21%

4 25 5.27% 29 6.00% 54 5.64%

5 173 36.50% 137 28.36% 310 32.39%

6 49 10.34% 32 6.63% 81 8.46%7 76 16.03% 88 18.22% 164 17.14%

8 150 31.65% 196 40.58% 346 36.15%

Total 474 100.00% 483 100.00% 957 100.00%

Education GenderFemale Male Total

3 1 1 20.21% 0.21%

50.00% 50.00%0.10% 0.10% 0.21%

4 25 29 545.27% 6.00%

46.30% 53.70%2.61% 3.03% 5.64%

5 173 137 31036.50% 28.36%55.81% 44.19%18.08% 14.32% 32.39%

6 49 32 8110.34% 6.63%60.49% 39.51%5.12% 3.34% 8.46%

7 76 88 16416.03% 18.22%46.34% 53.66%7.94% 9.20% 17.14%

8 150 196 34631.65% 40.58%43.35% 56.65%15.67% 20.48% 36.15%

Total 474 483 95749.53% 50.47% 100.00%

Count--AbsoluteFrequency

Education GenderFemale Male Total

3 1 1 20.21% 0.21%

50.00% 50.00%0.10% 0.10% 0.21%

4 25 29 545.27% 6.00%

46.30% 53.70%2.61% 3.03% 5.64%

5 173 137 31036.50% 28.36%55.81% 44.19%18.08% 14.32% 32.39%

6 49 32 8110.34% 6.63%60.49% 39.51%5.12% 3.34% 8.46%

7 76 88 16416.03% 18.22%46.34% 53.66%7.94% 9.20% 17.14%

8 150 196 34631.65% 40.58%43.35% 56.65%15.67% 20.48% 36.15%

Total 474 483 95749.53% 50.47% 100.00%

JointProbability

Education GenderFemale Male Total

3 1 1 20.21% 0.21%

50.00% 50.00%0.10% 0.10% 0.21%

4 25 29 545.27% 6.00%

46.30% 53.70%2.61% 3.03% 5.64%

5 173 137 31036.50% 28.36%55.81% 44.19%18.08% 14.32% 32.39%

6 49 32 8110.34% 6.63%60.49% 39.51%5.12% 3.34% 8.46%

7 76 88 16416.03% 18.22%46.34% 53.66%7.94% 9.20% 17.14%

8 150 196 34631.65% 40.58%43.35% 56.65%15.67% 20.48% 36.15%

Total 474 483 95749.53% 50.47% 100.00%

MarginalProbability

Education GenderFemale Male Total

3 1 1 20.21% 0.21%

50.00% 50.00%0.10% 0.10% 0.21%

4 25 29 545.27% 6.00%

46.30% 53.70%2.61% 3.03% 5.64%

5 173 137 31036.50% 28.36%55.81% 44.19%18.08% 14.32% 32.39%

6 49 32 8110.34% 6.63%60.49% 39.51%5.12% 3.34% 8.46%

7 76 88 16416.03% 18.22%46.34% 53.66%7.94% 9.20% 17.14%

8 150 196 34631.65% 40.58%43.35% 56.65%15.67% 20.48% 36.15%

Total 474 483 95749.53% 50.47% 100.00%

P(Ed =4|F)

P(F|Ed=4)

ConditionalProbabilities:

Education GenderFemale Male Total

3 1 1 20.21% 0.21%

50.00% 50.00%0.10% 0.10% 0.21%

4 25 29 545.27% 6.00%

46.30% 53.70%2.61% 3.03% 5.64%

5 173 137 31036.50% 28.36%55.81% 44.19%18.08% 14.32% 32.39%

6 49 32 8110.34% 6.63%60.49% 39.51%5.12% 3.34% 8.46%

7 76 88 16416.03% 18.22%46.34% 53.66%7.94% 9.20% 17.14%

8 150 196 34631.65% 40.58%43.35% 56.65%15.67% 20.48% 36.15%

Total 474 483 95749.53% 50.47% 100.00%

MarginalProbability

ConditionalProbabilities

JointProbability

AbsoluteFrequencies

Example 8--More Probability Calculations

Find the probability that the individual: is a high school graduate is female is male or has incomplete high school is female and did not complete college graduated from college given that he is

a male is male given that he graduated from

college

Next Time ...

Bayes Rule Total Probability Rule Applications