Probability and Independence 1 Krishna.V.Palem Kenneth and Audrey Kennedy Professor of Computing...

Probability and Independence

1

Krishna.V.PalemKenneth and Audrey Kennedy Professor of ComputingDepartment of Computer Science, Rice University

Probability

2

The classical definition of probability Pierre Simon

LaplaceThe probability of an event is the ratio of the number of cases favorable to it, to the number of all cases possible when nothing leads us to expect that any one of these cases should occur more than any other, which renders them, for us, equally possible.Let us elaborate…

Consider an event space

Event 1

Event 2

Event 3

Event 4

If there is no reasonbelieve that one event is more likely to occur

than another

Probabilityof favorableevents

Favorable events

No. of favorableevents

Total no. of events

Generalizing Probability

3

Consider another event space

Event 1

Event 2

Event 3

Event 4

Let us assume that each event is differently likelyto occur

Let us represent as a list of magnitude of“likeliness”

EventLikelines

s

Event 1 p1

Event 2 p2

Event 3 p3

Event 4 p4

Generalizing Probability (Contd.)

4

EventLikelines

s

Event 1 p1

Event 2 p2

Event 3 p3

Event 4 p4

Let us look at these values.

If these values have the following properties

1. All pi are in the range [0,1] 2. The sum of all pi = 1.

then these values are called the probabilities of theseevents.

For example, Event Probabiliti

es

Event 1 0.5

Event 2 0.3

Event 3 0.1

Event 4 0.1

Satisfies both the conditions

This means that event 1 occurs 1 out of 4 times … etc.

Specification

5

Consider the event space

Event 1

Event 2

Event 3

Event 4

Event Probabilities

Event 1 p1

Event 2 p2

Event 3 p3

Event 4 p4

Complete specification of the

behavior of the experiment

Both these together

6

Now we have this relationship

Event

Event 1

Event 2

Event 3

Event 4

Probabilities

p1

p2

p3

p4

This can be defined concisely as a function whose independent variablerepresents the event

The dependent variable is the valueof the probability

Let the variable ‘x’ represent the event

x Event

1Event

1

2Event

2

3Event

3

4Event

4

ipixp )(Probability (Event i)

Here ‘x’ is called a random variable.Where i={1,2,3,4}

Analysis of the event space of a coin tossCan you list the event space of an

experiment where an unbiased coin is being tossed ?

To complete the specification we need the probabilities of these events.

7

Event H

Event T

Event Probabilities

Event H ½

Event T ½

The method of union of eventsJoint Event space of

two coins

Event HH

Event TT

Event HTEvent TH

EventProbabili

ty

Event HH

1/4

Event HT

1/4

Event TH

1/4

Event TT

1/4

Consider the outcome

Heads in both the coins “OR”Tails in both the coins

From the previous definition,

Probability of the outcome = 2/4

= 1/2

Another way of representing the same probability is

Probability of (Heads in both the coins “OR” Tails in both the coins)

= Probability (Heads in both the coins) + Probability(Tails in both the coins)

Why do you think that the “OR” of two events is translated to a + for their probabilities ?8

Mutually Exclusive events and outcomes

Joint Event space of two coins

Event HH

Event TT

Event HTEvent TH

Two events are said to be mutually exclusive if

(i)Given one event as the outcome of the experiment it is assumed that all other outcomes are discarded.

For example, if Event HH is the outcome of the experimentthen Event TT “cannot” the outcome of the experiment

Let us consider another example,


First coin is H

Second coin is T

First coin is TSecond coin is H

The event space of the two coins can also be represented like this because itcovers all possible events

But these two events are not mutually exclusive because if the eventthat the first coin is H occurs then the event that the second coin isH can also occur

Hence these events are “not” mutually exclusive9

The method of union of events


Event HH

Event TT

Event HTEvent TH

The method of union of events states that “ If two events are mutually exclusive then the probabilitythat either of them will occur is the sum of probabilities of the two events”

10

Let us play the experiment ∞ times

Outcomes

HH HT TH TT HH HH TT HH HT TH HT TH TH …. ∞

Relative Frequency = (HH , HH, HH) + (TT,TT,TT,….)

All Outcomes All Outcomes

1/4 1/4+ = 1/2

For infinite trials For infinite trials


11

RecallConsider an event space

Event 1

Event 2Event 3

Event 4

Favorable events


No. of favorableevents

Total no. of events

Event Probabilities

Event 1 p1

Event 2 p2

Event 3 p3

Event 4 p4

….Etc.

Given all the above events are mutually exclusive


.......42 Etcpp

….. Etc.


Let us consider the case of two dice

Joint Event Space of Die 1 & 2

Event 11Event 12

Event 21Event 23

….. Etc.Using method of union of events, calculate the following

(i)Probability that the first DIE and the second die rolled a ‘3’ OR both the DICE rolled a ‘6’

A) P(Event 33 OR Event 66) = 1/36 + 1/36 = 1/18

(ii) Probability that 3 and 4 were rolled be either one of them

A) P(Event 34 OR Event 43) = 1/36 + 1/36 = 1/18

Are these events mutually exclusive ?

12

Mini-exercise 1 – Snakes and Ladders(S&L)Calculate the probability of landing in

square 6 OR square 8 in the following scaled-down version of snakes and ladders (no snake or ladder)?

13

7 8 9

6 5 4

1 2 3

We will start at square 1 Roll the die 50 times and

record the squares that you land in

Calculate the chance of landing in square 6 OR square 8

Note that favorable events could be derived both from landing on square 6 OR on square 8

Instructions for Mini-Exercise 1

The method of intersection of events


Event HH

Event TT

Event HTEvent TH

Event Space of coin 1

Event H

Event T


Event H

Event T

Event

P

H 2/3

T 1/3

Event

P

H ½

T ½

Consider this joint experimentagain

What is the probability of the outcome of the experiment having HEAD in the first die and TAIL in the second die ?

All the events are not equally likely

Cannot use the earlier definition ofFavorable events

Total number of events14



Event HH

Event TT

Event HTEvent TH


Event H

Event T


Event H

Event T

Event

P

H 2/3

T 1/3

Event

P

H ½

T ½

Consider this joint experimentagain

What is the probability of the outcome of the experiment having HEAD in the first coin and TAIL in the second coin ?

= Probability of first coin having HEAD * Probability of second coin having TAIL = ½ * 1/3 = 1/6

H in coin 1 = ½ of the time

T in coin 2= 1/3 of this = ½ * 1/3 = 1/6 of the time

15



Event HH

Event TT

Event HTEvent TH


Event H

Event T


Event H

Event T

Event

P

H 2/3

T 1/3

Event

P

H ½

T ½

The method of intersection of events says “The probability of a joint event is equal to the product of probability of elementary events”

16

The method of intersection of eventsLet us consider the case of two dice

Using method of intersection of events,Calculate the following

(i) Probability that the first die rolled a ‘1’ and the second die rolled a ‘4’

A) P(Event 14) = P(Event 1) * P(Event 4) = 1/12 * 1/12 = 1/144

(ii) Probability that the first die rolled a ‘6’ and the second die rolled a ‘6’

A) P(Event 66) = P(Event 6) * P(Event 6) = 1/3 * 1/12 = 1/36

Outcome

Die 1 Die 2

1 1/12 1/3

2 1/12 1/3

3 1/12 1/12

4 1/12 1/12

5 1/3 1/12

6 1/3 1/12

Table of probabilities of the outcomes

17

Mini-exercise 2 – Snakes and Ladders(S&L)Calculate the probability of landing in

square 2 AND square 8 in the following scaled-down version of snakes and ladders (no snake or ladder)?

18

7 8 9

6 5 4

1 2 3

We will start at square 1 Roll the die for 50 times

and record the squares that you land in

Calculate the chance of landing in square 6 AND 8 in one cycle

In this case, favorable events requires that both events occur

A cycle is completed when you go from square 9 to square 1

Instructions for Mini-Exercise 2

Building a modelAn important component of this course

(including exercises and projects) would be building models, and these games are an example.

An example of a model would be

19

4

5

6

1/61/6

20

Solution:Probability of reaching 6 via 5 = 1/6Probability of reaching 6 via 4 = 1/6

Total probability of reaching 6 from 4 OR 5 = 1/6 + 1/6 = 1/3

4

5

6

1/61/6

21

4

5

6

1/6

1/6

1/6

You are at square 4 and want to compute the probability of reaching square 6

This can happen in two ways

(i) You roll a 1 and reach 5 and then roll a 1 again

(ii) You roll a 2 and reach 6

As we know the probabilities of the rolls we can calculate the probability of reaching 6 from 4

Solution:Probability of reaching 6 via 5 = (1/6 * 1/6 )Probability of reaching 6 directly = 1/6

Total probability of reaching 6 from 4 = 7/36

Mini-Exercise 3 Can you build a model for the snakes and

ladder game till square 6 ?In this game, if we cross square 6 with a die

roll we stay at 6That means, if you are at square 4 and you roll a 5,

then you still stay at 6

22

6 5 4

1 2 3

Remember the elements of a modelA node or a circle that indicates a particular square in

the gameAn arrow or an edge that indicates the possibility of

reaching another square from the current squareA label on each arrow that indicates the probability of

that transition taking place

23

4

5

6

3

21

1/6

3/6

1/6

1/6

1/6

2/6

1/61/6

1/6

24

This is a partial model of the scaled down snakes and ladder game

4

5

6

5/6

1/6

6/6 3

21

1/6

1/6

3/6

1/6

1/6

4/6

1/6

1/6

2/6

1/61/6

Mini-Exercise - 4 Using the model of the snakes and ladders,

calculate the following valuesThe probability of reaching square 4 from

square 1

The probability of reaching square 5 OR square 6 from square 2

The probability of reaching square 3 AND square 5 from square 1

25

26

6 5 4

1 2 3

4

5

6

5/6

1/6

6/6 3

21

1/6

1/6

3/6

1/6

1/64/6

1/6

1/6

2/6

1/61/6

Let us start with the transition model we built

Hint: New edge(s) might be added as an extension

Mini-Exercise - 5 Using the new model of the snakes and

ladders with the snake, calculate the following valuesThe probability of reaching square 4 from

square 1

The probability of reaching square 5 OR square 6 from square 2

The probability of reaching square 3 AND square 5 from square 1

27

Question ?

28

4

5

6

1/6

1/6

1/6 3

2

1

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/61/6

Say that you are at square 4 at a point in the game

You roll a die and you reach 6

What can we do to get that information ?

29

4

5

6

1/6

1/6

1/6 3

2

1

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/61/6

What if we place some sort of a token at each square we land ?

Let us say that we land at square 2

And then we reached square 4

So we leave a token at square 2

Thus after completing the game we have the additional “information” to determine where we have been

More information

30

4

5

6

1/6

1/6

1/6 3

2

1

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/61/6

So basically if we play this game multiple times, then we need more tokens (probably with different colors) to keep track of where we were each time.

Let us say that an all-knowing “Oracle” tells us that the previous square we were in was odd and not even.

Do we still need tokens to keep track ?

Will we need more tokens or less ?

Answers

31

4

5

6

1/6

1/6

1/6 3

2

1

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/6

1/61/6

We will analyze more precisely the relationship between two events in the next class.

Independence and DependenceTill now, we have been looking at events individually. But in practice, events are interrelated. Some events are dependent on some other event

taking place

Let us consider the following example

Let us play a small game. Bob is rolling a die. He asks Alice to guess what he rolled.

1.What is the probability that Bob is correct ?

2.Let us say that the Oracle told Danny that the outcome of the die is an even number or an odd number. Then what is the probability that Danny is correct ?

32

There is an underlying mathematical concept that can be explicitly stated to calculatethe answer to the question

Consider an experiment.

The outcome of the experiment can be specified in terms of two different event spaces

Event 1Event 2Event 3

…

Event EvenEvent Odd

Knowledge of the outcome of the roll of a die in terms of whether it is an even number or an odd number allows us to predict the actual outcome more precisely

p

p1

p2

p3

…

p

Peve

n

Podd

33 Let us calculate how much it improves our quality of our guess

There is an underlying mathematical concept that can be explicitly stated to calculatethe answer to the question

Consider an experiment.

The outcome of the experiment can be specified in terms of two different event spaces

Event 1Event 2Event 3

…

Event EvenEvent Odd

Knowledge of the outcome of the roll of a die in terms of whether it is an even number or an odd number allows us to predict the actual outcome more precisely

p

p1

p2

p3

…

p

Peve

n

Podd

34 Let us calculate how much it improves our quality of our guess

Probability and Independence 1 Krishna.V.Palem Kenneth and Audrey Kennedy Professor of Computing...

Documents

Transcript of Probability and Independence 1 Krishna.V.Palem Kenneth and Audrey Kennedy Professor of Computing...