Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A B)...

46
Some Rules of Probability 1 0 A P 1 S P B P A P B A P B A then , If A P A P c 1 C P B P A P C B A P C B C A B A then , , If B A P B P A P B A P B A then , If B P A P B A P B A then t independen are , If t? independen not are , if What B A

Transcript of Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A B)...

Page 1: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Some Rules of Probability 10 AP 1SP

BPAPBAPBA then , If

APAP c 1

CPBPAPCBAP

CBCABA

then

,, If

BAPBPAPBAP

BA

then

, If

BPAPBAP

BA

then

t independen are , If

t?independennot

are , ifWhat BA

Page 2: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

More formulae:

• P(B|A) = =

Thus, P(B|A) is not the same as P(A|B).

• P(AB) = P(A|B)·P(B)

• P(AB) = P(B|A)·P(A)

CONDITIONAL PROBABILITYCONDITIONAL PROBABILITY

P(A)A)P(B

P(A)B)P(A

)()()|(

BPBAPBAP

Page 3: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

AIDS Testing Example

ELISA test: + : HIV positive

- : HIV negative

Correctness: 99% on HIV positive person (1% false negative)

95% on HIV negative person

(5% false alarm)

Mandatory ELISA testing for people applying for marriage licenses in MA.

“low risk” population: 1 in 500 HIV positive

Suppose a person got ELISA = +.Q: HIV positive?

Page 4: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Bayes’ Theorem

cc APABPAPABP

APABPBAP

||

||

…some peoplemake a living outof this formula

Try Michael Birnbaum’s (former UIUC psych faculty) Bayesian calculatorhttp://psych.fullerton.edu/mbirnbaum/bayes/BayesCalc.htm

Page 5: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Bayes’ Theorem

cc APABPAPABP

APABPBAP

||

||

notHIVPnotHIVPHIVPHIVP

HIVPHIVPHIVP

||

||

500

4995001

5001

05.99.

99.|

HIVP

038.| HIVP 99.| HIVP

Page 6: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

The Binomial DistributionThe Binomial Distribution

Today: Today:

Page 7: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

We have already talked about the most famous continuous random variable, which,

because it is so heavily used, even has a name:

The Normal Random Variable (and, associated with it, the Normal Distribution)

Today we will talk about a famous discrete random variable, which, because it is so heavily used, also has a name:

The Binomial Random Variable (and, associated with it, the Binomial Distribution)

Page 8: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

FAIR COIN“POPULATION”

THEORETICAL PROBABILITY OF HEADS IS ½

If you toss a fair coin10 times,

what is the probability of

x many heads?

Page 9: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.5, n=10

0

0.05

0.1

0.15

0.2

0.25

0.3

0 1 2 3 4 5 6 7 8 9 10

# of successes

pro

ba

bil

ity

Page 10: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Random Variable

Potential Examples:Repeat a simple dichotomous experiment n times and count…

Coin Tossing: # headsMarketing: # purchasesMedical procedure: # patients curedFinance: # days stock Politics: # voters favoring a given billTesting: # number of test items of a given difficulty

that you solve correctlySampling: # of females in a random sample of peopleEducation: # of high school students who drink alcohol

Page 11: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Population

Repeat simple experiment n many times independently.

)success" ofy probabilit" (e.g.,

parameter a is

10

1

:RV sdichotomou a is

p

pYP

pYP

Y

n

in

yyy

YYYYY

,...,,

:DataObtain

. ofcopy a is Each ,...,,

21

21

RV) (a statistic a is

...

successes) # (i.e., s1' # Let

21

X

YYYX

X

n

, ..., n, X 10 valuescan take

onDistributi Binomial

, ..., n, kkXP

a called is

10for

ondistributiy probabilit The

XonDistributi Sampling of theis This

Page 12: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

X: number of successes in n many independent repetitions of an experiment,each repetition having a probability p of success

Binomial Random Variable

),(~ pnBX

Page 13: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

The Binomial DistributionBinomial Distribution, p=.25, n=10

0

0.05

0.1

0.15

0.2

0.25

0.3

0 1 2 3 4 5 6 7 8 9 10

# of successes

pro

ba

bili

ty

Page 14: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Trial 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0

Trial 2 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0

Trial 3 1 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0

Trial 4 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0

Probab.

(letting

q=1-p)

pppp

pppq

ppqp

ppqq

pqpp

pqpq

pqqp

pqqq

qppp

qppq

qpqp

qpqq

qqpp

qqpq

qqqp

qqqq

X 4 3 3 2 3 2 2 1 3 2 2 1 2 1 1 0

Page 15: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Probab.

(letting

q=1-p)

pppp

pppq

ppqp

ppqq

pqpp

pqpq

pqqp

pqqq

qppp

qppq

qpqp

qpqq

qqpp

qqpq

qqqp

qqqq

X 4 3 3 2 3 2 2 1 3 2 2 1 2 1 1 0

44 pppppXP )1(4)1(43 3 ppppppXP

22 161162 ppppppXP 31411141 ppppppXP

4111110 pppppXP

Page 16: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

In general, for n many trials

knk ppkXP 1successesk get to waysof #

How do we figure that out

in general?

Page 17: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Factorial

!

!3

!5

n

12...1

123

12345

nn

Page 18: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution Formula:

1!0

)1)(2)...(2)(1)((! and

called )!(!

! where

)1()(

bygiven is ...,,1,0each for

successes)many exactly ( )(

nnnn

tCoefficien Binomialknk

n

k

n

ppk

nkXP

nk

kPkXP

knk

TABLE CPages T-6 to T-10

in the book

Page 19: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Probab.

(letting

q=1-p)

pppp

pppq

ppqp

ppqq

pqpp

pqpq

pqqp

pqqq

qppp

qppq

qpqp

qpqq

qqpp

qqpq

qqqp

qqqq

X 4 3 3 2 3 2 2 1 3 2 2 1 2 1 1 0

44 pXP

)1(43 3 ppXP

22 162 ppXP

3141 ppXP

410 pXP

knk ppk

nkXP

)1()(

Page 20: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

44 pXP

)1(43 3 ppXP

22 162 ppXP

3141 ppXP

410 pXP

1!0!4

!4:4 k

4

1123

1234

!34!3

!4:3

k

6

1212

1234

!24!2

!4:2

k

4

1231

1234

!14!1

!4:1

k

1!4!0

!4:0 k

knk ppk

nkXP

)1()( BINOMIAL COEFFICIENTS:

Page 21: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.5, n=5

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0 1 2 3 4 5

# of successes

pro

ba

bil

ity

Page 22: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.5, n=10

0

0.05

0.1

0.15

0.2

0.25

0.3

0 1 2 3 4 5 6 7 8 9 10

# of successes

pro

ba

bil

ity

Page 23: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.5, n=20

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# of successes

pro

ba

bil

ity

Page 24: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.5, n=50

0

0.02

0.04

0.06

0.08

0.1

0.12

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50

# of successes

pro

ba

bil

ity

Page 25: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.5, n=100

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.090 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 99

# of successes

pro

bab

ility

Page 26: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

)1(

for Variance and

Value Expected

2 pnp

np

X~B(n,p)

X

X

Example:

# heads in 5 tosses of a coin: X~B(5,1/2)

Expectation Variance# heads in 5 tosses of a coin: 2.5 1.25

Page 27: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

)1(

for Variance and

Value Expected

2 pnp

np

X~B(n,p)

X

X

PROOF:

YYYYYX in ofcopy t independenan is each where...21

pYPpYP

Y

10,1

:RV sdichotomou a is

pppppp

ppp

Y

Y

1101

101222

pnpn

npn

YYYYYYYX

YYYYYYYX

nn

nn

1...

...22222

...2

...

2121

2121

Page 28: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Another Statistic: The Sample Proportion

knk ppk

nkXP

)1()(Remember that X is a

random variable

The sample proportion is a linear transformation of X and thus a random variable too

kXn

kp

n

XXp n en exactly wh ˆ thusand ˆ 1

knk ppk

n

n

kpP

)1(ˆ

Sampling Distributionof the Sample Proportion:

Page 29: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

The Sample Proportion

knk ppk

nkXP

)1()(

knk ppk

n

n

kpP

)1(ˆ

)1(

for Variance and

Value Expected

2 pnp

np

X~B(n,p)

X

X

n

pp

n

pp

n

pnp

n

pn

X~B(n,p)n

Xp

pX

p

Xp

)1(,

)1()1(

whereˆfor Variance and

Value Expected

ˆ22

22ˆ

ˆ

Unbiased Estimator

Page 30: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Let’s take another look at some Binomial Distributions

especially what happens as n gets bigger and bigger

Page 31: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.1, n=5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 1 2 3 4 5

# of successes

pro

ba

bil

ity

Page 32: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.1, n=10

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0 1 2 3 4 5 6 7 8 9 10

# of successes

pro

ba

bil

ity

Page 33: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.1, n=20

0

0.05

0.1

0.15

0.2

0.25

0.3

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# of successes

pro

ba

bil

ity

Page 34: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.1, n=50

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50

# of successes

pro

ba

bil

ity

Page 35: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, p=.1, n=100

0

0.02

0.04

0.06

0.08

0.1

0.12

0.140 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 99

# of successes

pro

bab

ility

Page 36: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Normal Approximation of/to the Binomial

n

pppNp

pnpnpNpnBX

n

1,ely approximat is ˆ ly,consequent and

1,ely approximat is ,~

thenlarge, is If

Page 37: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

0

0.05

0.1

0.15

0.2

0.25

0.3p

rob

ab

ilit

y

Page 38: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Normal Approximation

1100

1 ppp

1100

1 ppp

p

1100

1,~ˆ

1100 :Example

pppNp

n

Page 39: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Normal Approximation: Let’s check it out!

789.1,420,20~

,20 :Example

54

51

51

51

NNX

pn

kk

kkXP

20

54

51

20

TABLE CPages T-6 to T-10

in the book

0020.10

2010

0222.8

208

0576.1

201

:exampleFor

10

5410

51

12

548

51

19

541

51

XP

XP

XP

Page 40: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Normal Approximation: Let’s check it out!

789.1,420,20~

,20 :Example

54

51

51

51

NNX

pn

?8 isWhat XP

8 :ionApproximat Normal theUsing XP

X

X

X

XXP

8

Standardizing:

23.2789.1

48

ZPZP

0129.23.21 ZP

Page 41: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Normal Approximation: Let’s check it out!

789.1,420,20~

,20 :Example

54

51

51

51

NNX

pn

?8 isWhat XP

0222.8 XP

8 :Table Binomial theUsing XP

0074.9 XP

0020.10 XP

0005.11 XP

0001.12 XP

0322.

...0001.

0005.0020.

0074.0222.8

XP

Page 42: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Normal Approximation: Let’s check it out!

789.1,420,20~

,20 :Example

54

51

51

51

NNX

pn

?8 isWhat XP

0322.8 :Table Binomial theUsing XP

0129.8 :ionApproximat Normal theUsing XP

Are westuck

with a badapproximation??

Page 43: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, n=20

0

0.05

0.1

0.15

0.2

0.25

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# of successes

pro

ba

bil

ity

0.2

0322.8

:onDistributi Binomial

for C Table Using

XP

Page 44: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, n=20

0

0.05

0.1

0.15

0.2

0.25

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# of successes

pro

bab

ility

0.2

0129.8

:ionApproximat Normal

theUsing

XP

Page 45: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

Binomial Distribution, n=20

0

0.05

0.1

0.15

0.2

0.25

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# of successes

pro

bab

ility

0.2

0250.9750.1

789.1

45.75.7

:Correction Continuitywith

ionApproximat Normal theUsing

ZPXP

Page 46: Some Rules of Probability. More formulae: P(B|A) = = Thus, P(B|A) is not the same as P(A|B). P(A  B) = P(A|B)·P(B) P(A  B) = P(B|A)·P(A) CONDITIONAL.

For now, that’s it …

We will revisit the Binomial:

• Based on the sample proportion as an estimate of the population proportion, we will develop “confidence intervals” for the population proportion.

• We will carry out hypothesis tests about the true population proportion, using the information gained from the sample proportion.