I. Basic Probability

Stat 5120 – Categorical Data Analysis

Dr. Corcoran, Fall 2013

I. Basic Probability

Readings for Section I: Rosner (supplementary handout), Chapter 3

An experiment is any exercise whose outcome is unknown in advance.

A sample space S represents all possible outcomes for the experiment.p

Example I.A

Experiment: Flip a fair coin. S = {H,T}.



Example I.B

i l i di id l f l i d bExperiment: Select an individual from a population and observe gender and smoking status. S = {female smoker, female nonsmoker, male smoker, male nonsmoker}., , }

Example I.C

Suppose a researcher is interested in sampling from a stand of trees, and then classifying individual trees according to type (ponderosa pine, Douglas fir, white fir, oak, or blue spruce) and presence or p , g , , , p ) pabsence of bark beetles. What is the sample space for this experiment?



Probability is a measure of the likelihood that a particular outcome occurs.

Probabilities for the outcomes of any experiment have to follow certain rules – that is, if an experiment has the sample space

},,,,{ 21 NOOOS

where the Oi represent the individual N possible outcomes, and the probability of Oi is given by P(Oi) = pi, then

and ,10 ip

N

i.

.1)(1

N

ii SPpii.



Example I.D

Experiment: flip a fair coin, with S = {H,T}. Note that P(H) = P(T) = ½. Hence, each probability lies between 0 and 1, inclusive, and P(S) = 1.

Example I.E

Suppose we sample an individual from the U S population and categorize themSuppose we sample an individual from the U.S. population, and categorize them according to race, so that S = {White, Black, Hispanic, Asian, Native American, Other}. If P(White) = 0.75, P(Black) = 0.12, P(Hispanic) = 0.06, P(Asian) = 0 04 what are the possible values of P(Native American)?0.04, what are the possible values of P(Native American)?

If P(Other) = 0.02, then what is P(Native American)?( ) , ( )



Regarding Our Choice of Examples

It is sometimes easiest to illustrate concepts of basic probability by using games of chance, such as flipping coins, rolling dice, or d i f d Th i t l ti l lldrawing face cards. These experiments are relatively well understood with probabilities that are easy to verify.

In addition, the study of modern probability began as people became interested in better understanding the randomness in games of chance.of chance.



How do we interpret probability?

For example, suppose we assume that there are roughly an equal number of men and women studying at USU, and that we conduct the following experiment: go down to the Quad and observe the gender of the next individual that passes by.

What is the probability that person is a woman? What does such aWhat is the probability that person is a woman? What does such a probability mean?

(HINT: the probability of an outcome is a reflection of the long-term relative frequency of that outcome.)



Events

i b f f i iAn event is a subset of outcomes from a given experiment. We usually denote an event by a capital letter, such as A, B, or C, for example. p

An event A consists of all outcomes in an experiment’s sample space S such that A is observed to occur if any one of thosespace S such that A is observed to occur if any one of those outcomes is observed to occur.

Th b bili f A d d b P(A) i h f hThe probability of an event A – denoted by P(A) – is the sum of the probabilities of all of the outcomes belonging to A.



Example I.F

We roll two fair six-sided dice. The sample space is given by:

),6,1(),3,1(),2,1(),1,1({

),6,3(),3,3(),2,3(),1,3(),6,2(),3,2(),2,2(),1,2(

S

})6,6(),3,6(),2,6(),1,6(

Let A {The sum of the two dice is prime}Let A = {The sum of the two dice is prime}.

What outcomes belong to A?

What is P(A)?



Event Complements

The complement of an event A is denoted by Ac or A’; it consists of all outcomes in S that do not belong to A.

Note: P(A) + P(A’) = 1.

Example I.G

Based on Example I.F, what outcomes belong to A’?Based on Example I.F, what outcomes belong to A ?

What is P(A’)?



Combinations of EventsIn research, we are often interest in how two or more events relate to one another. For example, suppose a scientist wants to assess how a behavior like smoking is associated with heart disease. She might pose questions such as:

i ll h i di id l k i hi h i i i• Proportionally how many individuals smoke within the investigative population? Proportionally how many have heart disease? How many either smoke or have disease?

h i h h h d l l d i di id l k d h• What is the chance that a randomly sampled individual smokes and has heart disease?

• Given that someone smokes, what’s the probability that he or she has heart disease?heart disease?

• Do these proportions depend further on other factors, such as gender or weight?

All of these questions involve probabilities of combinations of two or more events.



Definitions

Given two events A and B in a sample space S we are interestedGiven two events A and B in a sample space S, we are interested in the following combinations:

1. Event intersection (“A and B”), denoted bywith probability denoted by

,BA

2 Event union (“A and/or B”) denoted by BA

).( BAP

2. Event union ( A and/or B ), denoted bywith probability denoted by

,BA

)( BAP ).( BAP



Probabilities of Event Combinations

The probability is given by the sum of the probabilities of all outcomes)( BAP The probability is given by the sum of the probabilities of all outcomes belonging to both A and B. Likewise, the probability is given by the sum of the probabilities of all outcomes belonging to A and/or B.

)( BAP )( BAP

These combinations are sometimes best illustrated using a Venn diagram, as shown below; for example, in this diagram, we have shaded the subset of outcomes belonging to

.BA

BA



Probabilities of Event Combinations

The Venn diagram below makes it easy to see thatThe Venn diagram below makes it easy to see that

:)()()()( BAPBPAPBAP

Use a Venn diagram to show, for example, that g , p ,

'. ')'( BABA



Example I.H

i i i h l lContinuing with example I.F, let

A = {The sum of the two dice is prime}, and { p },B = {The sum of the two dice is odd}.

What is ? What is ?BA )( BAP What is ? What is ?BA )( BAP

What is ? What is ?BA )( BAP



Three or More Events

ll f h d ll bi i f hAll of these concepts extend naturally to combinations of three or more events. For example, based on the Venn diagram below, can you give a formula for that uses only probabilities )( CBAP y g y pof intersections?

)(



Conditional Probability

Some of the most compelling questions in research follow thisSome of the most compelling questions in research follow this pattern: “If X, then what of Y?” For example,

• If a patient with cancer is given this drug, what is his probabilityof recovery?

• Given that a survey participant is female, how will sheGiven that a survey participant is female, how will sherespond to this question?

Such questions can be assessed using conditional probabilitySuch questions can be assessed using conditional probability. Given two events A and B, the conditional probability of A given Bis defined as

)( .)(

)()|(BP

BAPBAP



Conditional Probability

Note that “A|B” is not a set of outcomes – in other words, A|B is not a combination of events, like an intersection or union. Hence, it only makes sense to discuss conditional probabilities notit only makes sense to discuss conditional probabilities – not conditional events. What we are asking is: “Does knowledge that B occurred give us additional information about the likelihood of A?”A?”

Example I.Ip

Consider again Example I.H. Without doing any computation, does knowing that the sum of two dice is odd change thedoes knowing that the sum of two dice is odd change the likelihood that the sum is also prime?



Example I.I (cont’d)

The somewhat obvious answer to the last question on the previous slide is “yes” – that is, all but one of the prime numbers between 2 and 12 (inclusive) are odd. Knowing that the sum is odd means that there’s a greater likelihood (compared to having no such information) that the sum is also prime. This is verified by using ) p y gthe definition of conditional probability already given:

736/14)( BAP .97

36/1836/14

)()()|(

BPBAPBAP

Note for this example that P(A|B) > P(A), which supports our intuition.



Conditional Probability and Event Intersections

Using simple algebra you can see that )|()()( ABPAPBAP Using simple algebra, you can see that

This can be extended ad infinitum:

).|()()( ABPAPBAP

etc. ),|()|()()( BACPABPAPCBAP

Example I.J

A deck of face cards has 4 suits (hearts, diamonds, clubs, and spades) and 13 denominations per suit (2 3 10 J Q K A) Hearts and diamonds are red suits anddenominations per suit (2, 3,…, 10, J, Q, K, A). Hearts and diamonds are red suits, and clubs and spades are black suits. Hence, there are 52 total cards – 26 red and 26 black. What is the probability that I draw three consecutive red cards (without replacement) from a shuffled deck?

Note that there are two ways of approaching this solution: we can either enumerate the entire sample space and count up the number of ways of drawing three reds, or we can apply the relationship shown above.pp y p



Independence

Two events A and B are said to by independent if P(A|B) = P(A). In other words, knowledge about B does not change the likelihood of A. In Example I.I, A and B are therefore said to be dependent.of A. In Example I.I, A and B are therefore said to be dependent.

By virtue of the formula shown above, if A and B are independent, thenthen

).()()( BPAPBAP

Important: “mutually exclusive” and “independent” are not synonymous! On the contrary, they are nearly antithetic. Why?



IndependenceIn research settings, independence is not something we can verify a priori using conditional probabilities (although we can assessa priori using conditional probabilities (although we can assess dependence within the gathered sample). Rather, independence is often conceptual. For example, do you believe that the following variables are independent?

• Political party affiliation of a husband and wife.p y

• Political party affiliation of next-door neighbors.

• Political party affiliation of randomly selected individuals from the state of Utah.



Example I.K

d id d b l h iIn a recent study of auto accidents and seatbelt use, among those wearing seatbelts it was found that the survival rate was 99.88%, whereas the survival rate among those not wearing belts was 99.02%. Of all of the accident victims studied 71 56% were wearing seatbeltsstudied, 71.56% were wearing seatbelts.

What is the probability that an individual in an auto accident is not wearing a belt and dies?belt and dies?

What is the probability that someone dies in an auto accident?

Given that an accident victim died, what is the probability that he or she was wearing a seatbelt?

Are the events of dying and wearing a seatbelt independent? Why or why not?



Bayes’ Rule

h l i h hi d i i i l fThe solution to the third question in Example I.K is an example of Bayes’ Rule:

)()|()|( APABP

This rule is useful in situations where information about P(B|A)

.)'()'|()()|(

)()|()|(APABPAPABP

APABPBAP

This rule is useful in situations where information about P(B|A), P(B|A’), and P(A) is known, but P(A|B) is desired.

Thi i i ll i i li iThis is especially true in screening applications.



Example I.L

A AIDS k t d f h i d ti d t b “99 7%An AIDS screen marketed for home use is advertised to be “99.7% accurate”. What does that mean? It actually means the following: if you have AIDS, this test will yield a positive result 99.7% of the time. This is called the sensitivity of the screen. The specificitygives you the proportion of true negatives, or the probability that the screen is negative if you don’t have AIDSthe screen is negative if you don t have AIDS.

Why is sensitivity in and of itself not all that helpful to the average ?consumer?

What is the probability of interest for someone who orders this screen?



Example I.L (cont’d)

AIDS prevalence in Utah is about 9 cases per 10000 residents. In the U.S., the prevalence is about 25 per 10000. Given an AIDS

h d i 99 7% i i i d 99 1% ifi iscreen that advertises 99.7% sensitivity and 99.1% specificity, what is the probability that a randomly selected Utahn for whom this screen is positive actually has AIDS?p y

What is this probability for a randomly selected individual from the U S population?the U.S. population?

This quantity is called the positive predictive value of a screen.



Counting

As we have seen in some of our examples, computing a probability is often a matter of counting the number of ways an event can

d th di idi b th t t l l t i th loccur, and then dividing by the total elements in the sample space.

For many applications, counting is a matter of understanding the structure of the problem, and then applying a few basic counting rules.



Multiplication Rule

Suppose that we have an experiment that takes place in K stagesSuppose that we have an experiment that takes place in K stages, where at the ith stage there are ni possible outcomes, for i = 1,…,K. Then there are

possible experimental outcomes.Knnn 21

possible experimental outcomes.

Example I.M

For lunch, you have 3 choices of entrée, 5 sides to choose from, and 4 desserts. How many possible meals lie before you?



PermutationsSuppose that we want to start a Justin Bieber fan club in this class. We need a president, a vice president, and a treasurer. How many different ways can we select people from hi l fill h i i ?this class to fill these positions?

The answer is an application of the multiplication rule.

When we are interested selecting k ordered or distinguishable objects from n total objects, this is referred to as a permutation.

Th b f h t ti i i bThe number of such permutations is given by

,)!(

!)1()2)(1(kn

nknnnnPnk

where n! is read as “n factorial”, with

.12)2)(1(! nnnn

By definition, 1! = 0! = 1.



CombinationsSuppose now that our Justin Bieber fan club will be led by a committee of three coequal members. How many such committees can be formed from this class?

One possible permutation is Curtis, Joan, and Sydney. However, note that (since the positions are coequal) this is essentially the same committee as that yielded by the permutation Sydney, Joan, and Curtis, or any other combination of those three.p y y, , , y

In other words, to count the number ways of selecting k indistinguishable or unordered objects from n total objects – referred to as combinations – we need to divide out from the t t l b f t ti th b f th t t i ltotal number of permutations the number of ways that we can permute a single combination. For example, there are (3)(2)(1) = 3! ways of ordering the trio of Curtis, Joan, and Sydney (or any other permutation), so the number of ways we can choose kunordered objects from n is given by j g y

.)!(!

!! knk

nkPC

nkn

k

This is alternatively denoted by .

kn



Example I.N

Here’s an added bonus to help you with your next trip to Vegas:Here’s an added bonus to help you with your next trip to Vegas:

Five-card draw is a game in which each player is dealt 5 face cards from a shuffled deck. Hands with combinations of cards that are relatively more rare are relatively more valuable. For example, a “pair” refers to a hand that has two cards with the samepair refers to a hand that has two cards with the same denomination, and three remaining cards that are random (i.e., have no other interesting combinations). A “three-of-a-kind” has three cards of the same denomination A “full house” is a handthree cards of the same denomination. A full house is a hand containing a pair and a three-of-a-kind.

What is the probability of drawing a pair? Three-of-a-kind? Full house?

I. Basic Probability

Documents

Transcript of I. Basic Probability