Introduction to Probabilities

Farrokh Alemi, Ph.D.Saturday, February 21, 2004

Updated by Janusz WojtusiakFall 2009

Probability can quantify how uncertain we are about a future event

Why measure uncertainty? To make tradeoffs among uncertain

events  To communicate about uncertainty

What is probability?

In the Figure, where are the events that are not “A”?

How to Calculate Probability?

AP(A)=

AP(A)=

Calculus of Probabilities Helps Us Keep Track of Uncertainty of Multiple

EventsJoint probability, probability of either event occurring, revising

probability after knew knowledge is available, etc.

Probability of One or Other Event Occurring

P(A or B) = P(A) + P(B) - P(A & B)

Example: Who Will Join Proposed HMO?

P(Frail or Male) = P(Frail) - P(Frail & Male) + P(Male)

Probability of Two Events co-occurring

Effect of New Knowledge

Conditional Probability

Example: Hospitalization rate of frail elderly

Sources of Data Objective frequency 

– For example, one can see out of 100 people approached about joining an HMO, how many expressed an intent to do so?  

Subjective opinions of experts  – For example, we can ask an expert to

estimate the strength of their belief that the event of interest might happen. 

Two Ways to Assess Subjective Probabilities

Strength of Beliefs – Do you think employees will join the

plan?  On a scale from 0 to 1, with 1 being certain, how strongly do you feel you are right? 

Imagined Frequency – In your opinion, out of 100 employees,

how many will join the plan? Uncertainty for rare,

one time events can bemeasured

Axioms are always met,but that we want

them to be followed

All Calculus of Probability is Derived from Three Axioms

1. The probability of an event is a positive number between 0 and 1

2. One event will happen for sure, so the sum of the probabilities of all events is 1

3. The probability of any two mutually exclusive events is the sum of the probability of each.

Probabilities provide a context in which beliefs

can be studied Rules of probability provide a

systematic and orderly method

Partitioning Leads to Bayes Formula

P(Joining) = (a +b) / (a + b + c + d)  P(Frail) = (a + c) / (a + b + c + d) P(Joining | Frail)  = a / (a + c) P(Frail  |  Joining) = a / (a + b) P(Joining | Frail)  = P(Frail  | Joining)  *  P(Joining) /  P(Frail)

Frail elderly

Not frail elderly

Total

Joins the HMO a b a + bDoes not join the HMO c d c + dTotal a + c b + d a + b + c + d

Bayes Formula

Odds Form of Bayes Formula

Posterior odds after review of clues =Likelihood ratio associated with the clues * Prior odds

Independence The occurrence of one event does

not tell us much about the occurrence of another

P(A | B) = P(A) P(A&B) = P(A) * P(B)

Example of DependenceP(Medication error ) ≠

P(Medication error| Long shift)

Suppose that one in every fifty patients in a clinic is diagnosed with

cancer.You know that all ten patients waiting before you in the line

have been diagnosed with cancer.

What is probability that you will be diagnosed with

cancer?

Independence SimplifiesCalculation of Probabilities

Joint probability can be calculated from marginal

probabilities

Conditional Independence

P(A | B, C) = P(A | C) P(A&B | C) = P(A | C) * P(B | C)

Conditional Independence versus Independence

P(Medication error ) ≠ P(Medication error| Long shift)

P(Medication error | Long shift, Not fatigued) = P(Medication error| Not fatigued)

Can you come up with other examples

Conditional Independence Simplifies

Bayes Formula

Example: What is the odds for hospitalizing a female frail

elderly?

Posterior odds of

hospitalization=

Likelihood ratio

associated with being frail elderly

*

Likelihood ratio

associated with being

female

* Prior odds of hospitalization

Likelihood ratio for frail elderly is 5/2 Likelihood ratio for Females is 9/10.  Prior odds for hospitalization is 1/2

Posterior odds of hospitalization=(5/2)*(9/10)*(1/2) = 1.125

Verifying Independence Reduce sample size and recalculate Correlation analysis Directly ask experts Separation in causal maps

Verifying Independence by Reducing Sample Size

P(Error | Not fatigued) = 0.50 P(Error | Not fatigued & Long shift) = 2/4 =

0.50

CaseMedication

error Long shift Fatigue1 No Yes No2 No Yes No3 No No No4 No No No5 Yes Yes No6 Yes No No7 Yes No No8 Yes Yes No9 No No Yes10 No No Yes11 No Yes Yes12 No No Yes13 No No Yes14 No No Yes15 No No Yes16 No No Yes17 Yes No Yes18 Yes No Yes

Verifying Conditional Independence Through

Correlations

Rab is the correlation between A and B Rac is the correlation between events A

and C Rcb is the correlation between event C

and B If Rab= Rac Rcb then A is independent of

B given the condition C

Verifying Independence Through Correlations

0.91 ~ 0.82 * 0.95 

Case Age BP Weight1 35 140 2002 30 130 1853 19 120 1804 20 111 1755 17 105 1706 16 103 1657 20 102 155

Rage, blood pressure  = 0.91Rage, weight  = 0.82

R weight, blood pressure  = 0.95

Rage, blood pressure =

0.91 ~ 0.82 * 0.95 =  Ra

g

e

,

w

e

i

g

h

t

*

  R

w

e

i

g

h

t

,

b

l

o

o

d

p

r

e

s

s

u

r

e

Verifying Independence by Asking Experts

Write each event on a 3 x 5 card Ask experts to assume a population where

condition has been met  Ask the expert to pair the cards if knowing

the value of one event will make it considerably easier to estimate the value of the other

Repeat these steps for other populations Ask experts to share their clustering Have experts discuss any areas of

disagreement Use majority rule to choose the final

clusters

Verifying Independence by Causal Maps

Ask expert to draw a causal map Conditional independence: A node that if

removed would sever the flow from cause to consequence

Blood pressure does not depend on age given weight

Probability of Rare Events Event of interest is quite rare (less

than 5%)– Because of lack of repetition, it is

difficult to assess the probability of such events from observing historical patterns. 

– Because experts exaggerate small probabilities, it is difficult to rely on experts for these estimates. 

Measure rare probabilities through time to the event

Examples for Calculation of Rare Probabilities

Probability = 1 / (1+time to event)

ISO 17799 word Frequency of event Calculation

Rare  probability

Negligible Once in a decade =1/(1+3650) 0.0003Very low 2-3 times every 5 years =2.5/(5*365) 0.0014

Low <= once per year =1/365 0.0027Medium <= once per 6 months =1/(6*30) 0.0056

High <= once per month =1/30 0.0333Very high => once per week =1/7 0.1429

Take Home Lessons Probability calculus allow us to keep

track of complex sequence of events Conditional independence helps us

simplify tasks Rare probabilities can be estimated

from time to the event