Introduction to Statistical Data Analysis Lecture 2 ...Introduction Conditional Probability...

IntroductionConditional Probability

Independent EventsIntersection of Events

Union of EventsBayes’ Theorem

Counting Principles

Introduction to Statistical Data AnalysisLecture 2: Probability

James V. Lambers

Department of MathematicsThe University of Southern Mississippi

James V. Lambers Statistical Data Analysis 1 / 47




Counting Principles

EventsTypes of ProbabilityProperties of Probability

Events

Informally, probability is the likelihood that a particular event will occur.

To be able to compute probabilities, though, we need precise definitionsof the concepts included in this informal definition.

I An experiment is a process of measuring or observing an activity forthe purpose of collecting data.

I An outcome is a result of an experiment.

I A sample space is a set of all possible outcomes of an experiment.

I An event is an outcome, or a set of outcomes, of interest.Mathematically, an event is a subset of the sample space.





Counting Principles


Types of Probability

Classical probability is the number of outcomes contained in an event,relative to size of sample space.

That is, if E is an event, and S is the sample space, then the probabilityof E , denoted by P(E ), is defined by

P(E ) =|E ||S |

,

where, for any set A, |A| denotes the cardinality of A, which is simply thenumber of elements contained in A.





Counting Principles


Example

Consider the result of rolling a single six-sided die, which is anexperiment.

The outcome is the number showing on the die after it is rolled.

The sample space is the set S = {1, 2, 3, 4, 5, 6}, which contains allpossible results of the die roll.

Examples of events would be “rolling a 6”, which is the set {6}, or“rolling an odd number”, which is the set {1, 3, 5}.

If E is the event “rolling a number higher than 4”, which is the set{5, 6}, then

P(E ) =|E ||S |

=2

6=

1

3.





Counting Principles


Empirical Probability

Empirical probability is defined to be the frequency of an event relative tothe number of observations.

The distinction between classical probability and empirical probability isbased on the distinction between an entire population and a sample ofthat population.

Classical probability describes the likelihood of an event, relative to anexhaustive set of all possible outcomes that is its sample space, whereasempirical probability has, as its sample space, the outcomes of arelatively small number of experiments.





Counting Principles


Example

An example of empirical probability would be the likelihood that aparticular train will be late.

It is not practical to measure this likelihood using classical probability,which would involve enumerating a very large set of combinations ofcircumstances that determine whether the train is late or not.

To use empirical probability, one could record whether the train was lateor not over a period of time, say several weeks.

Then, the sample space consists of those days on which the arrival timeof the train was recorded, and the event consists of those days on whichthe train arrived late.

If enough observations are made, then the empirical probability is likelyvery close to what the classical probability would be, if it could bemeasured.





Counting Principles


Subjective Probability

Finally, subjective probability is probability that is based on intuitionrather than experiments.

For example, suppose you are playing craps, and have won a few gamesin a row.

Technically, each game is independent of others, so all other things beingequal, there is a fixed (if unknown) probability of winning the next game.

However, as you continue to win, your subjective probability of winningthe next game will likely decrease, based on the saying that luck “has torun out some time”.





Counting Principles


Properties of Probability

Regardless of the type of probability that is being measured, there arecertain properties that the probability of an event E must satisfy.

1. P(E ) = 1 if the event E is certain to occur.

2. P(E ) = 0 if it is certain that E will not occur.

3. P(E ) must satisfy 0 ≤ P(E ) ≤ 1.

4. If E1,E2, . . . ,En are mutually exclusive events, meaning that no twoof these events can occur simultaneously, then

P(E1 ∪ E2 ∪ · · · ∪ En) = P(E1) + P(E2) + · · ·+ P(En) =n∑

i=1

P(Ei ).





Counting Principles


Complementary Probability

A consequence of the first and fourth properties is that if we denote byE ′ the complement of an event E , which consists of all outcomes in thesample space that are not contained in E , then

P(E ′) = 1− P(E ),

because either E or E ′ is certain to occur, due to all outcomes in thesample space belonging to one event or the other, but not both.





Counting Principles


Example

Let E be the event that the sun is going to rise tomorrow. As theoften-used quintessential certainty, it is safe to say that P(E ) = 1.

After limited experimentation, I believe it is equally safe to say that if Lis the event in which I will ever choose winning lottery numbers, thenP(L) = 0, and this will certainly be the case if I make the wise choice togive up on playing.

There is no circumstance under which an event can have a negativeprobability, or a probability greater than 1.

If A is the event that a student earns an A in a particular course, and B isthe event that they earn a B, and so on, then these events are mutuallyexclusive, since the student can only be assigned one grade. Therefore,

P(A ∪ B ∪ C ∪ D ∪ F ) = P(A) + P(B) + P(C ) + P(D) + P(F ).





Counting Principles

Simple and Conditional Probability

Simple probability, also known as prior probability, is probability that isdetermined solely from the number of observations of an experiment.

On the other hand, conditional probability, also known as posteriorprobability, is the probability that an event A will occur, given thatanother event B has already occurred. It is denoted by P(A|B); somesources use the notation P(A/B).

One can think of conditional probability as using a reduced sample space.When measuring P(A|B), one is not considering the whole of the samplespace from which A and B originate; instead, one is only considering thesubset B of that sample space, and then determining how many elementsof that subset also belong to A.





Counting Principles

Independent Events

Informally, two events A and B are said to be independent if neither oneis influenced by the other.

Mathematically, we say that A is independent of B if

P(A|B) = P(A).





Counting Principles

Example

Let A be the event that John is late for work, and B be the event thatJane, who has no connection to John whatsoever and in fact lives andworks in a different city from John, is late for work.

These two events are independent, so P(A|B) = P(A).

On the other hand, suppose John drives to work and that C is the eventthat there is a major traffic jam in his city.

This event, if it occurs, could cause him to be late for work, so P(A) isinfluenced by P(C ).

That is, P(A) is not the same as P(A|C ). On the other hand, B and Care independent, so P(B|C ) = P(B).





Counting Principles

Multiplication RuleMutually Exclusive Events

Intersection of Events

Let A and B be two events. Then, the joint probability of A and B,denoted by A ∩ B, is the event consisting of all outcomes that belong toboth A and B.

Since events are defined to be subsets of the sample space, the jointprobability of events is simply the intersection of the corresponding sets.





Counting Principles


Contingency Tables

Joint probabilities arise in contingency tables, which list the number ofoutcomes that correspond to each possible pairing of results of twoexperiments.

In a contingency table, each row corresponds to a value of one variable(that is, one possible result of an experiment), and each columncorresponds to a value of a second variable.

Then, the entry in row i , column j of the table is the number ofoutcomes corresponding to the ith value of the first variable and the jthvalue of the second.





Counting Principles


Example

Based on a survey of 100 adults, the following contingency table lists thejoint probabilities for each combination of values of two variables, whichare gender and choice of smartphone purchase.

Gender iPhone Samsung Neither TotalMale 16 18 14 48Female 20 16 16 52Total 36 34 30 100

From this table, it can be seen that if one respondent is randomly chosenfrom those surveyed, and if M is the event that the respondent is male,and I is the event that the respondent owns an iPhone, thenP(M ∩ I ) = 16/100 = 0.16, whereas P(M) = 0.48 and P(I ) = 0.36.





Counting Principles


Multiplication Rule

Using the concept of intersection of events, we can now give a simpleformula for conditional probability, based on the definition given earlier:

P(A|B) =P(A ∩ B)

P(B).

Combining this formula with the definition of independent events, itfollows that if A and B are independent events, then

P(A ∩ B) = P(A)P(B).

This formula is called the multiplication rule for independent events. Ifthe events A and B are dependent, then the multiplication rule takes adifferent form:

P(A ∩ B) = P(A|B)P(B).





Counting Principles


Having the Same Birthday

We will use the multiplication rule to compute the probability that out of23 people, at least 2 of them have the same birthday.

For simplicity, we work with a 365-day year. First, we note that theprobability that two people have different birthdays is 364/365, becauseonce the first person’s birthday is known, the second person’s birthdaycan fall on any one of the other 364 days.

Then, given that the first two people have different birthdays, theprobability that the third person has a different birthday is 363/365.





Counting Principles


Using the Multiplication Rule

Continuing this process, if we let Ai be the event that the ith person hasa different birthday than the first i − 1 people, the probability that all 23people have different birthdays is

P(A23 ∩ A22 ∩ · · · ∩ A2) = P(A2)P(A3|A2)P(A4|A2 ∩ A3) · · ·P(A23|A2 ∩ A3 ∩ · · · ∩ A22)

=364

365

363

365· · · 343

365= 0.493.

Therefore, the probability that at least two of the 23 people have thesame birthday is 1− 0.493 = 0.507. That is, there is a 50% chance thatat least two of them have the same birthday.





Counting Principles


Example

As before, let M be the event that a randomly chosen respondent ismale, and let I be the event that they own an iPhone. Then

P(M|I ) =P(M ∩ I )

P(I )=

0.16

0.36= 0.4.

This can also be seen by considering only the column of the table thatcorresponds to iPhone owners: there are 36 respondents who are iPhoneowners, and 16 of those are male, so based on that,P(M|I ) = 16/36 = 0.4.





Counting Principles


Example, cont’d

The table can be used to determine whether the events M and I areindependent. We know that P(M ∩ I ) = 0.16. From the totals of thefirst row and first column of the table, we have P(M) = 0.48 andP(I ) = 0.36. However, because

P(M)P(I ) = (0.48)(0.36) = 0.1728 6= P(M ∩ I ),

we conclude that these events are dependent.

On the other hand, suppose two six-sided die are rolled. The numbershown on each die is independent of the other, and since the probabilityof either die roll being a 6 is 1/6, we can conclude that the probability ofrolling double sixes is (1/6)(1/6) = 1/36.





Counting Principles


Interpretation of Conditional Probability

To reinforce the notion that conditional probability is the probability ofan event with respect to a reduced sample space, we note that if S is theoriginal sample space, then

P(A|B) =P(A ∩ B)

P(B)

=|A ∩ B|/|S ||B|/|S |

=|A ∩ B||B|

.

That is, P(A|B) is obtained by restricting the sample space to alloutcomes in B.





Counting Principles


Mutually Exclusive Events

Two events A and B are said to be mutually exclusive if it is not possiblefor A and B to occur simultaneously.

In set notation, we say that A and B are disjoint, or that A ∩ B = ∅.

Since there are no outcomes that belong to both A and B, it follows thatfor mutually exclusive events A and B,

P(A ∩ B) = 0.





Counting Principles

Addition Rule

Union of Events

The union of two events A and B is the event consisting of all outcomesthat belong to either A or B (and possibly both; the “or” is inclusive).

Using set notation again, we denote this event by A ∪ B.





Counting Principles

Addition Rule

Addition Rule

If two events A and B are mutually exclusive, then, from one of theproperties of probability stated earlier, it follows that

P(A ∪ B) = P(A) + P(B).

On the other hand, if A and B are not mutually exclusive, then the aboveformula does not hold, because outcomes that are in both A and B endup being counted twice.

Therefore, we need to correct the formula as follows:

P(A ∪ B) = P(A) + P(B)− P(A ∩ B).





Counting Principles

Addition Rule

Example

Consider the act of drawing a single card from a standard 52-card deck.

Let A be the event that the card drawn is a spade, let B be the eventthat the card drawn is a heart, and let C be the event that the carddrawn is a face card (jack, queen or king).

Then, the events A and B are mutually exclusive, but the events A andC are not, because it is possible to draw a jack, queen or king of spades.





Counting Principles

Addition Rule

Example, cont’d

From

P(A) = P(B) =1

4, P(C ) =

3

13, P(A ∩ C ) =

3

52,

we obtain

P(A ∪ B) = P(A) + P(B) =1

4+

1

4=

1

2,

and

P(A ∪ C ) = P(A) + P(C )− P(A ∩ C ) =1

4+

3

13− 3

52=

11

26.





Counting Principles

Bayes’ Theorem

Given two events A and B, Bayes’ Theorem is a result that relates theconditional probabilities P(A|B) and P(B|A).

It states that

P(B|A) =P(B)P(A|B)

P(B)P(A|B) + P(B ′)P(A|B ′).

To see why this theorem is true, note that by the multiplication rule, thenumerator on the right-hand side is simply P(A ∩ B), and thedenominator becomes P(A ∩ B) + P(A ∩ B ′).





Counting Principles

Bayes’ Theorem, cont’d

Because B and B ′ are mutually exclusive, but also exhaustive (meaningB ∪ B ′ is equal to the entire sample space), this expression becomesP((A ∩ B) ∪ (A ∩ B ′)) = P(A). We therefore have

P(B|A) =P(A ∩ B)

P(A),

which can be rearranged to again obtain the multiplication rule.





Counting Principles

Alternative Form of Bayes’ Theorem

Also, if we keep the original numerator in but use the simplifieddenominator, we obtain another commonly used statement of Bayes’Theorem,

P(B|A) =P(B)P(A|B)

P(A).

This form is very useful for computing one conditional probability fromanother that may be easier to obtain.





Counting Principles

Example

Suppose an insurance company classifies people as accident-prone or notaccident-prone.

Furthermore, they determine that the probability of an accident-proneperson actually having an accident within the next year is 0.4, whereasthe probability of a non-accident-prone person having an accident withinthe next year is 0.2.

If 30% of people are accident-prone, then what is the probability thatsomeone who does have an accident within the next year actually isaccident-prone?





Counting Principles

Applying Bayes’ Theorem

To answer this question, we let A be the event that the person has anaccident within the next year, and let B be the event that the person isaccident-prone.

From the given information, we have

P(A|B) = 0.4, P(A|B ′) = 0.2, P(B) = 0.3.

From these probabilities, we conclude that

P(A) = P(A|B)P(B) + P(A|B ′)P(B ′) = (0.4)(0.3) + (0.2)(0.7) = 0.26.

Using Bayes’ Theorem, we conclude that the probability of someone whohas an accident being accident-prone is

P(B|A) =P(B)P(A|B)

P(A)=

(0.3)(0.4)

0.26= 0.4615.





Counting Principles

The Fundamental Counting PrinciplePermutationsCombinationsPermutations and Combinations in R

Counting Principles

In order to compute probabilities using the definition, it is necessary to beable to determine the number of outcomes in an event or a sample space.

In this section, we present some techniques for counting how manyelements are in a given set.





Counting Principles


The Fundamental Counting Principle

The Fundamental Counting Principle states that if there are m ways toperform task A, and n ways to perform task B, then

I The number of ways to perform task A and task B is mn, and

I The number of ways to perform task A or task B (but not both) ism + n.





Counting Principles


Example

Suppose that an ice cream shop offers a selection of ten different flavors,five different toppings, and three different sizes.

Then the number of possible orders of ice cream is 10(5)(3) = 150.

On the other hand, suppose that at a particular restaurant, one entreeselection offers either steak or chicken, and a choice of a side dish.

If there are 7 different steak selections, 4 different chicken selections, and10 side dishes, then the number of possible variations of this entree are(7 + 4)10 = 110.





Counting Principles


Example

Standard license plates in California have a digit, followed by 3 letters,followed by another 3 digits.

Therefore, the number of possible license plates is

10 · 26 · 26 · 26 · 10 · 10 · 10 = 104263 = 175, 760, 000.

It can be seen from this example that if there are n ways to perform acertain task, and it must be performed r times, then the number of waysto do so is nr .





Counting Principles


Permutations

In many situations, it is necessary to know the number of possiblearrangements of things, or the number of ways to perform a task inwhich there is some sort of ordering.

Equivalently, it is often necessary to sample a number of objects in sucha way that (1) the order in which the objects are sampled is relevant, and(2) after an object is sampled, it is removed from the set so that itcannot be chosen again (this is known as sampling without replacement).

To see the equivalence, consider the task of arranging n objects. Once anobject is assigned its position, it should not be considered when placingthe second object, and then the second object should not be consideredwhen placing the third, and so on.





Counting Principles


Permutations, cont’d

To sample r objects, in order, from a set of n, without replacement, wefirst note that there are n ways to choose the first object.

Then, the chosen object is removed from consideration, meaning thatthere n − 1 ways to choose the second object.

Then, that object is removed consideration, leaving n − 2 ways to choosethe third object, and so on.





Counting Principles


Counting Permutations

Therefore, the number of ways to choose r objects from a set of n,without replacement, is

n(n − 1)(n − 2) · · · (n − r + 1) =n!

(n − r)!= nPr .

Since this is also the number of ways to arrange r objects chosen from aset of n, we call this the number of permutations of these objects.





Counting Principles


Example

Suppose that a club has 25 members, and it is necessary to elect apresident, vice-president, secretary, and treasurer.

Then, the number of ways to choose 4 members to fill these positions is

25P4 =25!

(25− 4)!= 25 · 24 · 23 · 22 = 303, 600.





Counting Principles


Example

We know from the Fundamental Counting Principle that the number ofpossible 4-letter words is 264.

This is in instance of sampling with replacement, because once the firstletter is chosen, it can be chosen again for the second letter, and so on.

However, if we require that all of the letters in each word are different,then we must sample without replacement, so the number of such wordsis 26P4 = 26(25)(24)(23).





Counting Principles


Combinations

It is often the case that a number of objects must be sampled withoutreplacement, but the order in which they are sampled is irrelevant.

In order to determine the number of ways in which such a sampling maybe performed, we can start by computing nPr , where n is the number ofobjects to choose from and r is the number of objects to be chosen, butthen we must divide by rPr = r !, the number of ways to arrange robjects.





Counting Principles


Binomial Coefficients

The result is

nCr =

(nr

)=

n!

r !(n − r)!,

which is called the number of combinations of r objects chosen from aset of n, also referred to as “n-choose-r”.

It is also the known as a binomial coefficient, as it arises naturally whencomputing powers of binomials.





Counting Principles


Example

Suppose we wish to count the number of possible poker hands.

This means counting the number of ways to choose 5 cards from a deckof 52.

The number in which the cards are chosen is irrelevant, so we usecombinations instead of permutations.

The number of hands is

52C5 =52!

5!(52− 5)!=

52 · 51 · 50 · 49 · 48

5 · 4 · 3 · 2 · 1= 2, 598, 960.





Counting Principles


Example

As another example, consider the 25-member club from the discussion ofpermutations.

Suppose that they need to form a 4-person committee.

The number of ways to do this is 25C4, the number of ways to choose 4members from a set of 25.

The reason why 25C4 is used here, as opposed to 25P4 for electing 4officers, is that a member’s position within the committee is irrelevant,whereas once 4 members are chosen to be officers, it matters which oneof them is chosen to be president, which is chosen to be vice-president,and so on.





Counting Principles


Permutations and Combinations in R

In R, the choose and factorial functions can be used to compute thequantities nPr and nCr .

To compute nCr , use choose(n,r).

To compute nPr , use choose(n,r)*factorial(r).





Counting Principles


Enumerating Combinations

The combn function can be used to actually enumerate all of thecombinations of elements of a vector.

For example, the combinations of 3 numbers chosen from the set{1, 2, 3, 4, 5} are

> v=c(1:5)

> combn(v,3)

[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]

[1,] 1 1 1 1 1 1 2 2 2 3

[2,] 2 2 2 3 3 4 3 3 4 4

[3,] 3 4 5 4 5 5 4 5 5 5


Introduction to Statistical Data Analysis Lecture 2 ...Introduction Conditional Probability...

Documents

Transcript of Introduction to Statistical Data Analysis Lecture 2 ...Introduction Conditional Probability...