Chapter 2
Probability
Sections 2.2-2.3
Sample Spaces and the Algebra of Sets
The Probability Function
“The most important questions of life are, for the most part, really only
problems of probability.”
Pierre Simon de Laplace, in his
"Theorie Analytique des Probabilites"
Casino: A House of Probability
What is Probability Theory?
Probability theory is the mathematics of randomness.
We shall be concerned with determining or quantifying the exact or estimated chance that a random event will occur.
Definition
An experiment is any procedure thatCan be repeated, theoretically, an infinite
number of timesHas a well-defined set of possible
outcomes
Examples of Experiments
Rolling a pair of dice Measuring a hypertensive’s blood
pressure Selecting a 3rd year BS ME major
Definition
Each possible result of an experiment is referred to as a sample outcome s. The set of all possible outcomes is called the sample space, and is usually denoted by S.
Examples
Roll a die and observe the number that comes up: S = {1,2,3,4,5,6}
Roll a die repeatedly and count the number of rolls it takes until the first 6 appears: S = {1,2,…}
Turn on a light bulb and measure its lifetime: S = [0,+)
Examples
Flip two coins and observe the sequence of heads and tails: S = {HH, TH, HT, TT}
Choose real numbers a, b, and c such that the equation ax2 + bx + c = 0 has imaginary roots: S = {(a,b,c)| b2 – 4ac < 0}
Example
(2.2.14) A probability-minded despot offers a convicted murderer a final chance to gain his release. The prisoner is given twenty chips, ten white and ten black. All twenty are to be placed into two urns, according to any allocation scheme the prisoner wishes, provided each urn has at least one chip. The executioner will then pick one of the two urns at random and from that urn, one chip at random. If the chip selected is white, the prisoner will be set free, otherwise, he “buys the farm.” Characterize the sample space describing the prisoner’s possible allocation options.
Definition
A subset A of the sample space S of an experiment is called an event.
Example
If we roll a die and observe the number that comes up, two possible events that can be defined are A: the outcome is odd, and B: the outcome is at least 4.
These can be viewed as subsets of the sample space S, with A = {1,3,5} and B = {4,5,6}.
Definitions
Let A and B be any two events defined over the same sample space S. Thena) The intersection of A and B, A B, is the event
whose outcomes belong to both A and B.
b) The union of A and B, A B, is the set of all outcomes in A or B (or both).
c) If A B = , then the events A and B are said to be mutually exclusive.
Remark 1
The notions of union and intersection can be extended to more than two sets. Let A1, A2,…, An be sets. We write
n
ini AAAA
121 ...
n
ini AAAA
121 ...
Remark 2
A useful property for sets is the Distributive Law:
A (B C) = (A B) (A C)
A (B C) = (A B) (A C)
Definition
Let A be any event defined on a sample space S. The complement of A, denoted by AC, is the set of outcomes in S not in A.
Remark 1
By De Morgan’s Law in logic, we can write the complement of a union or intersection of two events more simply as follows:
(A B)C = AC BC
(A B)C = AC BC
Remark 2
We can express unions, intersections, and complements in terms of Venn diagrams:
This can be done even for three or more events.
A BA B AC
Example 1
Let A1, A2,…, Ak be any set of events defined on a sample space S. Determine the outcomes that belong to the event CkCC
k AAAAAA ...... 2121
Example 2
Suppose that the events A1, A2,…, Ak are intervals of real numbers such that
Ai = {x| 0 x < 1/i}, i = 1,2,…,k.
Describe the sets and . What
happens as k ?
k
iiA
1k
iiA
1
Example 3
An internist has 520 patients, of which (1) 230 are hypertensive, (2) 185 are diabetic, (3) 35 are hypochondriac and diabetic, (4) 25 are all three, (5) 150 are none, (6) 140 are only hypertensive, and finally, (7) 15 are hypertensive and hypochondriac but not diabetic. How many of the internist’s patients are hypochondriac, but neither diabetic nor hypertensive?
Definitions of Probability
Classical Probability Empirical Probability Axiomatic Probability
Classical Probability
Suppose that an experiment has n possible outcomes, each outcome being equally likely to occur. If some event A were satisfied by m of these n, then the probability of A is m/n.
Empirical Probability
We can estimate the probability of an event A by repeating the same experiment over and over (say n times), and observing the number of times n(A) the event occurs.
The probability of A is then defined as
.n
Ann
)(lim
Definition (Probability Axioms)
Suppose that to each event A of a sample space S, a number denoted by P(A) is associated with A. If P satisfies the following axioms, then it is called a probability function and the number P(A) is said to be the probability of A. P(A) 0 P(S) = 1 If A1, A2, A3, … is a sequence of pairwise disjoint
events (that is, Ai Aj = for i j), then
11 kk
kk APAP
Some Implications
Let P be a probability function. Thena) P() = 0
b) If A1, A2,…, An is a sequence of pairwise disjoint events, then
n
kk
n
kk APAP
11
Theorem
Let P be a probability function on a sample space S and let A and B be events defined on S. Then
a) P(AC) = 1 – P(A)
b) P(A B) = P(A) + P(B) – P(A B)
c) If A B, then P(A) P(B).
d) P(A) 1.
Example 1
(2.3.1) According to a family-oriented lobbying group, there is too much crude language and violence on television. Forty-two percent of the programs they screened had language they found offensive, 27% were too violent, and 10% were considered excessive in both language and violence. What percentage of programs did comply with the group’s standards?
Example 2
(2.3.11) If State’s football team has a 10% chance of winning Saturday’s game, a 30% chance of winning two weeks from now, and a 65% chance of losing both games, what are their chances of winning exactly once?
Example 3
Let A and B be two events. Show that P(A)+P(B)–1 P(A B) P(A)+P(B).
Exercises
Ex. 2.2, #s 3,6,13 Ex. 2.3, #s 2,12,13,16
Section 2.4
Conditional Probability
Idea of Conditional Probability
Obtaining partial information about the outcome of a random experiment may, in many instances, change the probabilities of events.
A Simple Illustration
Suppose you roll a fair die and observe the number that faces up.
Obviously, the probability that the number that comes up is a 3 is 1/6.
However, if we are given that the result is odd, the above probability becomes 1/3.
A Simple Illustration
Define the ff. events:
T=“The number that comes up is a 3”
O=“The number that comes up is odd” Then P(T|O)=1/3, read as “the probability
of rolling a 3 given that an odd number is rolled.
A Simple Illustration
Note that P(T|O) = n(TO)/n(O) Dividing the numerator and denominator
by n, the number of sample points, gives
P(T|O) = P(TO)/P(O). Generalizing to any two such events leads
to the following definition.
Definition
Let A and B be any two events defined on a sample space S such that P(B)>0. Then the conditional probability of A given that B has occurred is
P(A|B) = P(AB)/P(B).
Example 1
(2.4.2) Find P(AB) if P(A)=0.2, P(B)=0.4, and P(A|B)+P(B|A)=0.75.
Example 2
(2.4.12) A fair coin is tossed three times. What is the probability that at least two heads will occur given that at most two heads have occurred?
Example 3
(2.4.18) Two fair dice are rolled. What is the probability that the sum of the two dice is greater than or equal to eight, given that at least one of the dice is a 5?
Theorem (Generalized Multiplicative Rule)
If in an experiment, the events A1, A2,…,Ak can occur, then
P(A1A2…Ak) = P(A1) P(A2|A1)
P(A3|A1A2) … P(Ak|A1A2…Ak-
1)
Example
Three cards are drawn in succession, without replacement, from an ordinary deck of playing cards. Find the probability that the first card is a red ace, the second card is a ten or jack, and the third card is greater than 3 but less than 7.
Theorem (Law of Total Probability)
Let A be an event with P(A)>0 and P(AC)>0. Then for any event B,
P(B) = P(B|A)P(A) + P(B|AC)P(AC)
Example
An insurance company rents 35% of the cars for its customers from agency I and 65% from agency II. If 8% of the cars of agency I and 5% of the cars of agency II break down during the rental periods, what is the probability that a car rented by this insurance company breaks down?
Definition
Let {A1,A2,…,An} be a set of nonempty subsets of the sample space S of an experiment. If the events A1,A2,…,An
are mutually exclusive and ,
the set {A1,A2,…,An} is called a partition of S.
SAn
ii
1
Theorem (Law of Total Probability)
If {A1,A2,…,An} is a partition of the sample space of an experiment and P(Ai) > 0 for i=1,2,…,n, then for any event B of S,
n
iii APABPBP
1)()|()(
Example
Suppose that 80% of the seniors, 70% of the juniors, 50% of the sophomores, and 30% of the freshmen of a college use the library of their campus frequently. If 30% of all students are freshmen, 25% are sophomores, 25% are juniors, and 20% are seniors, what percent of all students use the library frequently?
Theorem (Bayes’ Theorem)
Let {A1,A2,…,An} be a partition of the sample space S of an experiment. If for for i=1,2,…,n, P(Ai) > 0, then
n
iii
kkk
APABP
APABPBAP
1)()|(
)()|()|(
Example 1
(2.4.46) Brett and Margo have each thought about murdering their rich Uncle Basil in hopes of claiming their inheritance a bit early. Hoping to take advantage of Basil's predilection for immoderate desserts, Brett has put rat poison in the cherries flambe; Margo, unaware of Brett's activities, has laced the chocolate mousse with cyanide. Given the amounts likely to be eaten, the probability of the rat poison being fatal is 0.60; the cyanide, 0.90. Based on the other dinners where Basil was presented with the same dessert options, we can assume that he has a 50% chance of asking for the cherries flambe, a 40% chance of ordering the chocolate mousse, and a 10% chance of skipping dessert altogether. No sooner are the dishes cleared away when Basil drops dead. In the absence of any other evidence, who should be considered the prime suspect?
Example 2
A box contains seven red and 13 blue balls. Two balls are selected at random and are discarded without their colors being seen. If a third ball is drawn randomly and observed to be red, what is the probability that both of the discarded balls were blue?
Example 3
Example 3
(Monty Hall problem) On the game show Let’s Make a Deal, the host, Monty Hall, gives you a choice of three doors. Behind one door is the Grand Prize; behind the others, worthless prizes, called “zonks”. You pick a door, say Door A, and the host, who knows what is behind each door, opens another door (say Door B), revealing a zonk. The host then offers you the opportunity to change your selection to the remaining unopened door (say Door C). Should you stick with your original choice or switch? Does it make any difference?
Remark
This problem was published as a letter to Marilyn vos Savant’s “Ask Marilyn” column in Parade magazine in 1990.
Marilyn's response caused an avalanche of correspondence, mostly from people who would not accept her solution. Eventually, she issued a call to Math teachers among her readers to organize experiments and send her the charts. Some readers with access to computers ran computer simulations, which validated her answer.
Exercises
Ex. 2.4, #s 4, 9, 11, 19, 24, 29, 37, 41, 45, 53
Section 2.5
Independence
Definition
Two events A and B are said to be independent if P(AB) = P(A)P(B).
Example 1
Suppose we roll two dice (one red, one green) and observe the numbers that face up. Define the following events:
A: “The red die shows an even number of spots.”
B: “The number of spots on the two dice have the same parity.”
Are events A and B independent?
Example 2
Suppose that A and B are independent events. Show that AC and BC are also independent. (In fact, A and BC can also be shown to be independent.)
Definition
Events A1, A2,…, An are said to be independent if for every set of indices i1, i2,…, ik between 1 and n, inclusive,
kk iiiiii APAPAPAAAP ......
2121
Example
(2.5.14) In a roll of a pair of fair dice (one red and one green), let A be the event that the red die shows a 3, 4, or 5; let B be the event that the green die shows a 1 or 2; and let C be the event the dice total is 7. Show that A, B, and C are independent.
Probabilities of Intersections
Independence of events implies that the probability of one event is not affected by the probabilities of the other events.
In many cases, independence of events A1, A2,…, An follows immediately from physical considerations. In these cases, the definition provides an easy method of evaluating probabilities of intersections of several events.
Example 1
(2.5.4) Urn I has three red chips, two black chips, and five white chips; urn II has two red, four black, and three white. One chip is drawn at random from each urn. What is the probability that both chips are the same color?
Example 2
(2.5.6) Three points, X1, X2, and X3, are chosen at random in the interval (0,a). A second set of three points, Y1, Y2, and Y3, are chosen at random in the interval (0,b). Let A be the event that X2 is between X1 and X3. Let B be the event that Y1 < Y2 < Y3. Find P(A B).
Example 3
(2.5.22) According to an advertising study, 15% of television viewers who have seen a certain automobile commercial can correctly identify the actor who does the voiceover. Suppose that 10 such people are watching TV and the commercial comes on. (a) What is the probability that at least one of them can name the actor? (b) What is the probability that exactly one can name the actor?
Example 4
(2.5.27) An urn contains w white chips, b black chips, and r red chips. The chips are drawn out at random, one at a time, with replacement. What is the probability that a white appears before a red?
Exercises
Ex. 2.5, #s 2,5,19,24,26 Two one-peso coins, one with P(Head)=p and
one with P(Head)=q, are to be tossed together independently. Define
p0 = P(0 heads occur)p1 = P(1 head occurs)p2 = P(2 heads occur)
Can p and q be chosen such that p0 = p1 = p2? Justify your answer.
Sections 2.6, 2.7
Combinatorics
Combinatorial Probability
Recall: Classical Probability
Suppose that an experiment has n possible outcomes, each outcome being equally likely to occur. If m of these n satisfy an event A, then P(A)=m/n.
Counting Techniques
Fundamental Principle of Counting or Multiplication Rule
Permutations Combinations
Fundamental Principle of Counting If operation Ai, i=1,2,…,k can be
performed in ni ways, i=1,2,…,k, respectively, then the ordered sequence (operation A1, operation A2,…, operation Ak) can be performed in n1n2…nk ways.
Example 1
(2.6.4) Suppose that the format for license plates in a certain state is two letters followed by four numbers.(a) How many different plates can be made?(b) How many different plates are there if the letters can be repeated but no two numbers can be the same?(c) How many different plates can be made if repetitions of numbers and letters are allowed except that no plate can have four zeros?
Example 2
(2.6.9) A restaurant offers a choice of four appetizers, fourteen entrees, six desserts, and five beverages. How many different meals are possible if a diner intends to order only three courses? (Consider the beverage to be a “course.”)
Permutations
An ordered arrangement of r objects from a set A containing n objects (0rn) is called a permutation of the elements of A taken r at a time, and is denoted by nPr.
Theorem
The number of permutations of length r that can be formed from a set of n distinct elements is
where n! = n(n-1)(n-2)…(2)(1). (Note that as a convention, 0! = 1.)
)!(
!
rn
nPrn
Corollary
The number of ways to permute an entire set of n distinct objects is nPn = n!.
Example 1
(2.6.31) The crew of Apollo 17 consisted of a pilot, a copilot, and a geologist. Suppose that NASA had actually trained nine aviators and four geologists as candidates for the flight. How many different crews could they have assembled?
Example 2
(2.6.32) Uncle Harry and Aunt Minnie will both be attending your next family reunion. Unfortunately, they hate each other. Unless they are seated with at least two people between them, they are likely to get into a shouting match. The side of the table at which they will be seated has seven chairs. How many seating arrangements are available for those seven people if a safe distance is to be maintained between your aunt and your uncle?
Example 3 (Birthday Problem)
Suppose that k people are selected at random from the general population. What are the chances that at least two of those k were born on the same day?
Remark:
The following are the values of the probability for k=15, 23, 30, 40, 60: k P(at least two have same birthday) 15 0.253 23 0.507 30 0.706 40 0.891 60 0.995
Permutations of Non-distinct Objects The number of ways to arrange n
objects, n1 being of one kind, n2 of a second kind,…, nr of an rth kind, is
where .
!!...!
!
21 rnnn
n
r
ii nn
1
Example 1
(2.6.36) An interior decorator is trying to arrange a shelf containing eight books, three with red covers, three with blue covers, and two with brown covers. Assuming the titles and sizes of the books are irrelevant, in how many ways can she arrange the eight books?
Example 2
A delivery truck has to go from point X to point Y and make a stop at point O. How many different routes are possible, assuming the driver never wants to go out of her way?
Example 3
(2.7.12) If the letters in the phrase
A ROLLING STONE GATHERS NO MOSS
are arranged at random, what are the chances that not all the S’s will be adjacent?
Example 4
(2.6.46) Show that (k!)! is divisible by (k!)(k-1)!.
Exercises
Ex. 2.6, #s 6, 8, 9, 17, 23, 26, 29 Ex. 2.7, #s 10, 11, 14
Combinations
A selection of r objects from a set A containing n objects (0rn) without regard to order is called a combination of the elements of A taken r at a time, and is denoted by or nCr.
r
n
Theorem
The number of ways to form combinations of size r from a set of n distinct objects, where repetitions are not allowed, is given by
)!(!
!
rnr
n
r
nCrn
Example 1
In how many ways can two math and three biology books be selected from eight math and six biology books?
Example 2
A poker hand consists of 5 cards from a standard deck of 52. Find the number of different poker hands that are full houses. (Note: A full house consists of three cards of one denomination and two of another.)
Example 3
(2.7.4) A bridge hand (13 cards) is dealt from a standard 52-card deck. Let A be the event that the hand contains four aces; let B be the event that the hand contains four kings. Find P(A B).
Example 4
(2.6.58) Prove that
.n
n
nnn2...
10
Remark
The previous example implies that the number of subsets of a set with n (distinct) elements is 2n.
Example 5
(Vandermonde’s Identity) Use a combinatorial argument to show that for all positive integers m, n, and r,
r
i r
nm
ir
n
i
m
0
Exercises
Ex. 2.6, #s 37, 42, 51, 53, 56, 59