PROBABILITY AND PROBABILITY AND STATISTICSSTATISTICSPROBABILITYPROBABILITY

Probability

I Introduction to ProbabilityA Satisfactory outcomes vs. total outcomesB Basic PropertiesC Terminology

II Combinatory ProbabilityA The Addition Rule – “Or”

• The special addition rule (mutually exclusive events)• The general addition rule (non-mutually exclusive events)

B The Multiplication Rule – “And”• The special multiplication rule (for independent events)• The general multiplication rule (for non-independent events)

3

OBJECTIVES

develop sound understanding of probability values used in subsequent chapters develop basic skills necessary to solve simple probability problems

4

Event - any collection of results or outcomes from some procedure

Simple event - any outcome or event that cannot be broken down into simpler components

Sample space - all possible simple events

Experiment - a procedure that yields one of a given set of possible outcomes.

Definitions

Basic Properties of ProbabilitiesProperty 1: The probability of an event is always between 0 and 1,

inclusive.

Property 2: The probability of an event that cannot occur is 0. (An

event that cannot occur is called an impossible event.)

Property 3: The probability of an event that must occur is 1. (An

event that must occur is called a certain event.)

Probability for Equally Likely Outcomes

Suppose an experiment has N possible outcomes, all equally likely. Then the

probability that a specified event occurs equals the number of ways, f , that the event can

occur, divided by the total number of possible outcomes. In symbols:

Probability of a given event =

Number of ways a given event can occur

Total of all possible outcomes

7

Notation

P – denotes a probability

A, B, ... – denote specific events

P(A) – denotes the probability of event A occurring

8

Rule 1: Relative Frequency Approximation

Conduct (or observe) an experiment a large number of times, and count the number of times event A actually occurs, then an estimate of P(A) is

P(A) = number of times A occurred

number of times trial was repeated

Basic Rules for Computing Probability

9

Basic Rules for Computing ProbabilityRule 2: Classical approach

(requires equally likely outcomes)

If a procedure has n different simple events, each with an equal chance of occurring, and s is the number of ways event A can occur, then

P(A) = number of ways A can occurnumber of different simple

events

sn =

10

Rule 3: Subjective Probabilities

P(A), the probability of A, is found by simply guessing or estimating its value based on knowledge of the relevant circumstances.

Basic Rules for Computing Probability

11

Rule 1 The relative frequency approach is

an educated approximation.

Rule 2

The classical approach is the actual probability.Rule 3

The relative frequency approach is an absolute guess.

12

Law of Large Numbers

As a procedure is repeated again and again, the relative frequency probability (from Rule 1) of an event tends to approach the actual probability.

Two computer simulations of tossing a balanced coin 100 times

Law of Large Numbers

14

The sample space consists of two simple events: the person is struck by lightning or is not. Because these simple events are not equally likely, we can use the relative frequency approximation (Rule 1) or subjectively estimate the probability (Rule 3). Using Rule 1, we can research past events to determine that in a recent year 377 people were struck by lightning in the US, which has a population of about 274,037,295. Therefore,

Example: Find the probability that a randomly selected person will be struck by

lightning this year.

P(struck by lightning in a year) ≈ 377/274,037,295 ≈ 1/727,000

15

Example: On an ACT or SAT test, a typical multiple-choice question has 5 possible answers. If you make a random guess on one such question, what is the probability that your response is wrong?

There are 5 possible outcomes or answers, and there are 4 ways to answer incorrectly. Random guessing implies that the outcomes in the sample space are equally likely, so we apply the classical approach (Rule 2) to get:

P(wrong answer) = 4/5 = 0.8

16

Probability Limits The probability of an impossible event is 0.

The probability of an event that is certain to occur is 1.

0 ≤ P(A) ≤ 1

Impossibleto occur

Certainto occur

17

Possible Values for Probabilities

Certain

Likely

50-50 Chance

Unlikely

Impossible

1

0.5

0

18

Discrete ProbabilityDiscrete Probability

Example I: An urn contains four blue balls and five red balls. What is the probability that a ball chosen from the urn is blue?

Solution: There are nine possible outcomes, and the event “blue ball is chosen” comprises four of these outcomes. Therefore, the probability of this event is 4/9 or approximately 44.44%.

19

Discrete ProbabilityDiscrete Probability

Example II: What is the probability of winning the lottery 6/49, that is, picking the correct set of six numbers out of 49?

Solution: There are C(49, 6) possible outcomes. Only one of these outcomes will actually make us win the lottery.

p(E) = 1/C(49, 6) = 1/13,983,816

20

What happens if the outcomes of an experiment are notnot equally likely?

In that case, we assign a probability p(s) to each outcome s∈S, where S is the sample space.

Two conditions have to be met:

(1): 0 ≤ p(s) ≤ 1 for each s∈S, and

(2): ∑s∈S p(s) = 1

This means, as we already know, that (1) each probability must be a value between 0 and 1, and (2) the probabilities must add up to 1, because one of the outcomes is guaranteedguaranteed to occur.

DISCRETE DISCRETE PROBABILITYPROBABILITY

21

How can we obtain these probabilities p(s) ?

The probability p(s) assigned to an outcome s equals the limit of the number of times s occurs divided by the number of times the experiment is performed.

Once we know the probabilities p(s), we can compute the probability of an event E as follows:

p(E) = ∑s∈E p(s)

DISCRETE DISCRETE PROBABILITYPROBABILITY

22

Example I: A die is biased so that the number 3 appears twice as often as each other number.

What are the probabilities of all possible outcomes?

Solution: There are 6 possible outcomes s1, …, s6.

p(s1) = p(s2) = p(s4) = p(s5) = p(s6) and p(s3) = 2p(s1)

Since the probabilities must add up to 1, we have:

5p(s1) + 2p(s1) = 1 7p(s1) = 1

p(s1) = p(s2) = p(s4) = p(s5) = p(s6) = 1/7, p(s3) = 2/7

DISCRETE DISCRETE PROBABILITYPROBABILITY

23

Example II: For the biased die from Example I, what is the probability that an odd number appears when we roll the die?

Solution:

Eodd = {s1, s3, s5}

Remember the formula p(E) = ∑s∈E p(s).

p(Eodd) = ∑s∈Eodd p(s) = p(s1) + p(s3) + p(s5)

p(Eodd) = 1/7 + 2/7 + 1/7 = 4/7 = 57.14%

DISCRETE DISCRETE PROBABILITYPROBABILITY

24

COMPLEMENTARY EVENTS

P(A)or P(~A)(read “not

A”)

The complement of event A, denoted by A, or ~A consists of all outcomes in which event A does NOT occur.

25

Because 20 of the 50 subjects are men, it follows that 30 of the 50 subjects are women so,

P(not selecting a man) = P(~man)

= P(woman)

= 30/50 = 0.6

Example: Testing CorvettesThe General Motors Corporation wants to conduct a test of a new model of Corvette. A pool of 50 drivers has been recruited, 20 of whom are men. When the first person is selected from this pool, what is the probability of not getting a male driver?

26

Complimentary EventsComplimentary Events

Let E be an event in a sample space S. The probability of an event ~E, the complimentary event of E, is given by

p(~E) = 1 – p(E).

This can easily be shown:

p(~E) = (|S| – |E|)/|S| = 1 – |E|/|S| = 1 – p(E).

This rule is useful if it is easier to determine the probability of the complimentary event than the probability of the event itself.

27

Example I: A sequence of 10 bits is randomly generated. What is the probability that at least one of these bits is zero?

Solution: There are 210 = 1024 possible outcomes of generating such a sequence. The event ~E, “none of the bits is zero”, includes only one of these outcomes, namely the sequence 1111111111.

Therefore, p(~E) = 1/1024.

Now p(E) can easily be computed as p(E) = 1 – p(~E) = 1 – 1/1024 = 1023/1024.

COMPLIMENTARY COMPLIMENTARY EVENTSEVENTS

28

Example II: What is the probability that at least two out of 36 people have the same birthday?Solution: The sample space S encompasses all possibilities for the birthdays of the 36 people, so |S| = 36536.Let us consider the event ~E (“no two people out of 36 have the same birthday”). ~E includes P(365, 36) outcomes (365 possibilities for the first person’s birthday, 364 for the second, and so on). Then p(~E) = P(365, 36)/36536 = 0.168,so p(E) = 0.832 or 83.2%

COMPLIMENTARY COMPLIMENTARY EVENTSEVENTS

29

Odds actual odds against event A

occurring is the ratio P(~A)/P(A), usually expressed in the form of a:b (or a to b), where a and b are integers with no common factors

actual odds in favor of event A are the reciprocal of the odds against that event, P(A)/P(~A) or b:a (or b to a)

30

Odds

The payoff odds against event A represent the ratio of net profit (if you win) to the amount of the bet.

Payoff odds against event A =

(net profit):(amount bet)

Frequency distribution of annual income for U.S. families

Probability from Frequency Distributions

What is the probability of having an income between $15,000 and $24,999

10,040/69,597 = 0.1443

Frequency distribution for students’ ages

N = 40

What is the likelihood of randomly selecting a student who is 21?

What is the likelihood of selecting a student who’s age is an odd number?

What is the likelihood of selecting a student who is either 21 or 23?

7/40 = 0.175

21/40 = 0.525

10/40 = 1/4 = 0.25

Sample space for rolling a die once

Possible outcomes for rolling a pair of dice

Probabilities of 2 throws of the die

What is the probability of a 1 and a 3?

What is the probability of two sixes?

What is the probability of at least one 3?

2/36 or 1/18

1/36

12/36 or 1/3

1 2 3 64 5

1 2 3 64 5 1 2 3 64 5 1 2 3 64 5 1 2 3 64 5 1 2 3 64 5 1 2 3 64 5

TREE DIAGRAMThe probability of rolling a 1

and a 3.

Represents two paths out of thirty six possible paths.

2/36 or 1/18

The Sum of Two Die Tosses

What is the probability that the sum will be

5?

7?

What is the probability that the sum will be 10 or more?

What is the probability that the sum will be either 3 or less or 11 or more?

4/36 or 1/9

6/36 or 1/6

3/36 + 3/36 = 6/36 or 1/6

6/36 or 1/6

A deck of playing cards

A gambling experimentEveryone in the room takes 2 cards from the deck

(keep face down)

Rules, most to least valuable:

Pair of the same color (both red or both black)

Mixed-color pair (1 red, 1 black)

Any two cards of the same suit

Any two cards of the same colorIn the event of a tie, highest card wins (ace is top)

What do you want to bet?

Look at your two cards.

Will you fold or bet?

What is the most rational strategy given your hand?

Rational strategy

There are N people in the room

What are the chances that someone in the room has a better hand than you?

Need to know the probabilities of different scenarios

The Probability that the king of hearts is selected

1/52

The probability that a king is selected

4/52 = 1/13

13/52 = 1/4

THE PROBABILITY THAT A HEART IS SELECTED

The probability that a face card is selected

12/52 = 3/13

Venn diagram for event E

Relationships Among Events

(~ E): The event that “E does not occur.”

(A & B): The event that “both A and B occur.”

(A or B): The event that “either A or B or both occur.”

If E is the probability of drawing a face card, P(~E).

P(~E) = 40/52=10/13

But recall, P(E) = 12/52 = 3/13

and 1 – 3/13 =

13/13 – 3/13 = 10/13

An event and its complement

The Complementation Rule

For any event E, P(E) = 1 – P(~E)

In words, the probability that an event occurs equals 1 minus the probability that it does not occur

or P(~E) = 1 – P(E)

In words, the probability that an event does not occur equals 1 minus the probability that it does occur.

Combinations of Events

The Addition Rule – “Or”

The special addition rule (mutually exclusive events)

The general addition rule (non-mutually exclusive events)

The Multiplication Rule – “And”

The special multiplication rule (for independent events)

The general multiplication rule (for non-independent events)

Venn diagrams for (a) event (~E )(b) event (A & B) (c) event (A or B)

P(King & Heart) A King of Hearts

1/13 X 1/4 = 1/52

P(Face Card & Heart) A Face Card that is a Heart

3/13 X 1/4 = 3/52

Mutually Exclusive Events

Two or more events are said to be mutually exclusive if at most one of them can occur when the experiment is performed, that is, if no two of them have outcomes in common

Two mutually exclusive events

(a) Two mutually exclusive events(b) Two non-mutually exclusive events

(a) Three mutually exclusive events (b) Three non-mutually exclusive events (c) Three non-mutually exclusive events

The Special Addition Rule

If event A and event B are mutually exclusive, then

P(A or B) = P(A) + P(B)

More generally, if events A, B, C, … are mutually exclusive, then

P(A or B or C . . .) = P(A) + P(B) + P(C) + . . .

That is, for mutually exclusive events, the probability that at least one of the events occurs is equal to the sum of the individual probabilities.

Non-mutually exclusive events

The General Addition RuleIf A and B are any two events,

thenP(A or B) = P(A) + P(B) – P(A

& B)or

p(A ∪ B) = p(A) + p(B) - p(A ∩ B).

In words, for any two events, the probability that one or the other occurs equals the sum of the individual probabilities less the probability that both

occur.

P(king or heart) A King or a Heart

4/52 + 13/52 – 1/52 = 16/52 = 4/13

P(spade or face card)

P (spade) + P (face card) – P (spade & face card)

= 1/4 + 3/13 – 3/52= 22/52 = 11/26

65

Example: What is the probability of a positive integer selected at random from the set of positive integers not exceeding 100 to be divisible by 2 or 5?

Solution: E2: “integer is divisible by 2” ; E5: “integer is divisible by 5”

E2 = {2, 4, 6, …, 100} |E2| = 50 p(E2) = 0.5

E5 = {5, 10, 15, …, 100} |E5| = 20 p(E5) = 0.2

E2 ∩ E5 = {10, 20, 30, …, 100} |E2 ∩ E5| = 10 p(E2 ∩ E5) = 0.1

p(E2 ∪ E5) = p(E2) + p(E5) – p(E2 ∩ E5 )

p(E2 ∪ E5) = 0.5 + 0.2 – 0.1 = 0.6

THE GENERAL ADDITION RULE

The Special Multiplication Rule (for independent events)If events A, B, C, . . . are

independent, then

P(A & B & C & ...) = P(A) • P(B) •

P(C) • ...

What is the probability of all of these events occurring:

Flip a coin and get a headDraw a card and get an aceThrow a die and get a 1

P(A & B & C ) = P(A)· P(B)· P(C) = 1/2 X 1/13 X 1/6 = 1/156

67

Conditional ProbabilityConditional Probability

If we toss a coin three times, what is the probability that an odd number of tails appears (event E)(event E),, if the first toss is a tail (event F)(event F) ?

If the first toss is a tail, the possible sequences are TTT, TTH, THT, and THH.

In two out of these four cases, there is an odd number of tails.

Therefore, the probability of E, under the condition that F occurs, is 0.5.

We call this conditional probabilityconditional probability.

Conditional Probability: For non-independent events

The probability that event B occurs given that event A has

occurred is called a conditional probability. It is denoted by the symbol P(B | A), which is read

“the probability of B given A.” We call A the given event.

69

Example: What is the probability of a random bit string of length four contains at least two consecutive 0s, given that its first bit is a 0 ?

Solution:

E: “bit string contains at least two consecutive 0s”

F: “first bit of the string is a 0”

We know the formula p(E | F) = p(E ∩ F)/p(F)p(E | F) = p(E ∩ F)/p(F).

E ∩ F = {0000, 0001, 0010, 0011, 0100}

p(E ∩ F) = 5/16 p(F) = 8/16 = 1/2

p(E | F) = (5/16)/(1/2) = 5/32 = 0.15625

CONDITIONAL CONDITIONAL PROBABILITYPROBABILITY

Contingency table for age and rank of faculty members (using frequencies)

The Conditional-Probability RuleIf A and B are any two events,

then

In words, for any two events, the conditional probability

that one event occurs given that the other event has occurred equals the joint

probability of the two events divided by the probability of

the given event.

The Conditional-Probability Rule

P(R3 |A4) = 36/253 = 0.142P(A4 |R3) = 36/320 =

0.112

What is the probability that someone is an Assistant Professor given that they

are 50-59 years old?

What is the probability that someone is 50-59 years old given that they are an

Assistant Professor?

Joint probability distribution (using proportions)

P(R3 |A4) = 0.031/0.217 = 0.142

P( A4 | R3 ) = 0.031/0.275 = 0.112

What is the probability that someone is an Assistant Professor given that they

are 50-59 years old?

What is the probability that someone is 50-59 years old given that they are an

Assistant Professor?

Contingency table of marital status and sex (using proportions)

Joint probability distribution (using proportions)

The General Multiplication Rule If A and B are any two events, then

P(A & B) = P(A) • P(B | A).In words, for any two events, their joint probability equals the probability that

one of the events occurs times the conditional probability of the other

event given that event.Note: Either 1) The events are independent and

then P(A & B) = P(A) · P(B).

Or2) The events are not independent

and then a contingency table must be used

77

IndependenceIndependence

Let us return to the example of tossing a coin three times.

Does the probability of event E (odd number of tails) dependdepend on the occurrence of event F (first toss is a tail) ?

In other words, is it the case thatp(E | F) ≠ p(E) ?

We actually find that p(E | F) = 0.5 and p(E) = 0.5,so we say that E and F are independent eventsindependent events.

78

Because we have p(E | F) = p(E ∩ F)/p(F),p(E | F) = p(E) if and only if p(E ∩ F) = p(E)p(F).

Definition: Definition: The events E and F are said to be independent if and only if p(E ∩ F) = p(E)p(F).

Obviously, this definition is symmetricalsymmetrical for E and F. If we have p(E ∩ F) = p(E)p(F), then it is also true that p(F | E) = p(F).

INDEPENDENCINDEPENDENCEE

Independent EventsEvent B is said to be independent of event A if the occurrence of event A does not affect the probability that

event B occurs. In symbols,

P(B | A) = P(B).This means that knowing whether event A has occurred provides no probabilistic

information about the occurrence of event B.

Class Fr So Ju Se

Male 40 50 50 40| 180

Female 80 100 100 80 | 360

120 150 150 120 | 540

80

Bernoulli TrialsBernoulli Trials

Suppose an experiment with two possible two possible outcomesoutcomes, such as tossing a coin.

Each performance of such an experiment is called a Bernoulli trialBernoulli trial.

We will call the two possible outcomes a successsuccess or a failurefailure, respectively.

If p is the probability of a success and q is the probability of a failure, it is obvious that p + q = 1.

81

Often we are interested in the probability of exactly k exactly k successessuccesses when an experiment consists of n n independent Bernoulli trialsindependent Bernoulli trials.

Example:Example: A coin is biased so that the probability of head is 2/3. What is the probability of exactly four heads to come up when the coin is tossed seven times?

BERNOULLI BERNOULLI TRIALSTRIALS

82

Solution: Solution:

There are 27 = 128 possible outcomes.

The number of possibilities for four heads among the seven trials is C(7, 4).

The seven trials are independent, so the probability of each of these outcomes is(2/3)4(1/3)3.

Consequently, the probability of exactly four heads to appear is

C(7, 4)(2/3)4(1/3)3 = 560/2187 = 25.61%

BERNOULLI BERNOULLI TRIALSTRIALS

Probability and the Normal Distribution

What is the probability of randomly selecting an individual with an I.Q. between 95 and 115? Mean 100, S.D. 15.

Find the z-score for 95 and 115 and compute the area between

Probability and Experimental Design Using probability to evaluate a treatment effect. Values that are

extremely unlikely to be obtained from the original population are viewed as evidence of a treatment effect.

A Preview of Sampling Distributions

X X

What is the probability of randomly selecting a sample of three individuals, all of whom have an I.Q. of 135 or more?

Find the z-score of 135, compute the tail region and raise it to the 3rd power.

This concept is critical to understanding future concepts

So while the odds of a chance selection of a single person this far above the mean is not all that unlikely, the odds of a sample this far

above the mean are astronomical

z = 2.19P =

0.01430.01433 = 0.0000029

SummaryFor multiple events there are two rules:

“AND” (multiplication) and “OR” (addition)

There are just a few special considerations:

For the “And” rule, if the events are not independent, you don’t multiply, you use a table.

For the “Or” rule, if the events are not mutually exclusive you have to subtract off their double count