Post on 20-Feb-2022
Department of Computer Engineering, Faculty of Engineering,Kasetsart University, THAILAND
1st Semester 2019 (July – Nov)
Assoc. Prof. Anan Phonphoem, Ph.D.
Department of Computer Engineering, Faculty of Engineering,Kasetsart University, THAILAND
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 3
P[☺]is a function that maps event
in the sample space to real numberFrom experiment: Roll a diceOutcomes:
number = 1,2,3,4,5,6Sample space:
S = {1,2,3,…,6}Event examples:
E1 = {number < 3} = {1,2}E2 = {number is odd} = {1,3,5}
P[E1] = 2/6 = 1/3P[E2] = 3/6 = 1/2
Axiom 1: For any event A, P[A] ³ 0
Axiom 2: P[S] = 1
Axiom 3: For events A1, A2,…, Anof mutually exclusive events P[A1ÈA2È…ÈAn] = P[A1]+P[A2]+…+P[An]
4Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
• If we know P[A] before an experiment• P[A] » 1• Advanced knowledge à almost certainly occur
• P[A] » 0• Advanced knowledge à almost certainly not occur
• P[A] » ½• Advanced knowledge à maybe occur
• P[A] is a priori probability of A
5Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
• In practice, it maybe impossible to find the precise outcome of an experiment • However, if we know that Event B has
occurred• Probability of A when B occurs can be described
(the outcome of Event A is in set B) • Still don’t know P[A]
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 6
• Notation: P[A|B]• “Probability of A given B”• The condition probability of the event A
given the occurrence of the event B • Definition:
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 7
P[A|B] = P[AB]P[B]
• P[B] > 0
B
AA Ç B
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 8
S
A
BP[A|B] = P[AB]
P[B]
P[A|S] = P[AS]P[S]P[A]
1=
= P[A]
S
A
B
A
A Ç BB
A Ç B
S
A
BBA Ç B
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 9
Assume: all teams are equally likely to win the game
What is the probability that France will be the champion ?
132
What is the probability that Germany will be the champion, given that Sweden and Mexico are withdrawn ?
130
or 116
Let B1, B2,…,Bn be mutually exclusive events whose union equals sample space S à partition of S
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 10
B1 B2B3
B4 Bn…A
Si = 1
nP[A Ç Bi]P[A] =
For any event AA = AÇS = AÇ(B1È B2È…ÈBn) P[A] = P[AÇB1] + P[AÇB2] +…+ P[AÇBn]
Theorem:
• Let B1, B2,…,Bn be mutually exclusive events whose union equals sample space S• P[Bi] > 0
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 11
B1 B2B3
B4 Bn…A
P[A] = P[AÇB1] + P[AÇB2] +…P[A] = P[A|B1]P[B1] + P[A|B2]P[B2] +…
Si = 1
nP[A Ç Bi]P[A] = Theorem:
Si = 1
nP[A|Bi]P[Bi]P[A] = Theorem:
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 12
P[B|A] = P[BA]P[A]P[A|B]P[B]
P[A]=
P[A|B] = P[AB]P[B]
P[B|A]Theorem: P[A|B]P[B]P[A]=
www.pr-owl.org/basics/probability.php
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 14
Definition: Event A and B are independent iffP[AB] = P[A]P[B]
P[A|B] = P[AB]P[B]
= P[A]P[B]P[B]
P[B|A] = P[B]
P[A|B] = P[A]
P[A] = 0.3P[A|B] = 0.3
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 15
No matter event B occurs or not, event A is not affected
16
Independent Disjoint
P[AB] ¹ 0 P[AB] = 0
P[AÇB] = P[A]*P[B] P[AÈB] = P[A]+P[B]
BA
BA
Note: Independent = Disjoint iff P[A]=0 or P[B]=0
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 17
• 3 traffic lights, observe a sequence of lights• Mapping real world à (Simple) Model
1 2 3
redorgreen
• The sequence is equally likely• R1 = The first light was red• R2 = The second light was red• G2 = The second light was green• Are Event R2 and G2 independent?• Are Event R1 and R2 independent?
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 18
1 2 3
• 3 traffic lights, observe a sequence of lights
• Sample Space:• S = {rrr, rrg, rgr, rgg, grr, grg, ggr, ggg}
• Are Event R2 and G2 independent?
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 19
rrrrrggrrgrg
rgrrggggrggg
• P[R2]= P[{rrr, rrg, grr, grg}] = 4/8 = ½• P[G2]= P[{rgr, rgg, ggr, ggg}] = ½• P[R2G2] = 0• P[R2]P[G2] = (½)* (½) = ¼àR2 and G2 are not independentàR2 and G2 are disjoint
rrrrrggrrgrg
R2
rgrrggggrggg
G2
1 2 3
• P[R1]= P[{rrr, rrg ,rgr, rgg}] = ½• P[R2]= P[{rrr, rrg ,grr, grg}] = ½• P[R1R2] = P[{rrr ,rrg}] = 2/8 = 1/4• P[R1]P[R2] = (½) * (½) = ¼àR1 and R2 are independentàR1 and R2 are not disjoint
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 20
rrrrrggrrgrg
rgrrggggrggg
rrrrrggrrgrg
R2
R1rrrrrg
rgrrgg
• Are Event R1 and R2 independent?
21
Definition: Event A1,A2 and A3 are independent iff1) A1 and A2 are independent2) A2 and A3 are independent3) A1 and A3 are independent4) P[A1ÇA2ÇA3] = P[A1] P[A2] P[A3]
Definition: Event A and B are independent iffP[AB] = P[A] P[B] Is it sufficient ?
P[ABC] = P[A] P[B] P[C]
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
NO !
Note:
• Given: P[A] = P[B] = P[C] = 1/5• P[AB] = P[AC] = P[BC] = P[ABC] = 1/25• Independence in pairs (number 1-3) may not
independent
• Only number 4) is insufficient to guarantee the independence
• Ex.: One of the event is Null
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 22
• Assume that the event of separate experiments are independent• Example:• Assume that outcome of a coin toss is independent of
the outcomes of all prior and all subsequent coin tosses• P[H] = P[T] = ½• Toss a coin 3 times• S = • P[HTH] = 1/8• P[HTH] = P[H] P[T] P[H] = 1/2*1/2*1/2 = 1/8
{HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 23
HTH
• Experiment: in sequence sub-experiments à sub-experiments
• Each sub-experiment may depend on the previous one• Represented by a Tree Diagram• Model Conditional Prob. à Sequential Experiment
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 25
Leaf1 Outcome (node)
Outcomes ofthe completeExperiment (Leaf)
BranchProb. value
• Timing coordination of 2 traffic lights• P[the 2nd light is the same color as the 1st light] = 0.7• Assume 1st light is equally likely to be green or red
• Find P[The 2nd light is green] ?• Find P[wait for at least one light] ?
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 26
27
• P[G1] = P[R1] = 0.5
0.5
0.5
0.7
0.3
0.3
0.7
G1
R1
G2
G2
R2
R2
G1G2 = 0.35
G1R2 = 0.15
R1G2 = 0.15
R1R2 = 0.35
• P[G2G1] = P[G2|G1]P[G1] = (0.7)(0.5) = 0.35
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
28
P[The 2nd light is green] ?
P[G2] = P[G2G1] + P[G2R1] = 0.35 + 0.15 = 0.5
P[G2] = P[G2|G1]P[G1] + P[G2|R1]P[R1]
0.5
0.5
0.7
0.3
0.3
0.7
G1
R1
G2
G2
R2
R2
G1G2 = 0.35
G1R2 = 0.15
R1G2 = 0.15
R1R2 = 0.35
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
29
P[wait for at least one light] ?
W = {G1R2 È R1G2 È R1R2}
P[W] = P[G1R2] + P[R1G2] + P[R1R2] = 0.15 + 0.15 + 0.35 = 0.65
0.5
0.5
0.7
0.3
0.3
0.7
G1
R1
G2
G2
R2
R2
G1G2 = 0.35
G1R2 = 0.15
R1G2 = 0.15
R1R2 = 0.35
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 30
Japan
W1
L1
Colombia
D10.1
0.4
0.5
Senegal
W2
L2
D2
0.3
0.1
0.6
Poland
W3
L3
D3
W3
L3
D3
W3
L3
D3
0.6
0.1
0.30.6
0.1
0.30.6
0.1
0.3
W2
L2
D2
W2
L2
D2
0.3
0.1
0.6
0.3
0.1
0.6
W1W2W3 = 0.4 * 0.3 * 0.6 = 0.072W1W2D3 = 0.4 * 0.3 * 0.1 = 0.012W1W2L3 = 0.4 * 0.3 * 0.3 = 0.036
32
If experiment A has n possible outcomes,and experiment B has k possible outcomes,
->Then there are nk possible outcomes when you perform both experiments
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
33
1st draw: select 1 out of 52 -> 52 outcomes2nddraw: select 1 out of 51 (one card has been drawn)
-> 51 outcomes3rddraw: select 1 out of 50 -> 50 outcomes
Total outcomes = (52)(51)(50)
Example: Shuffle a deck and select 3 cards in order.How many outcomes?
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 34
(n-k)!(n-k)!
(n)k = n!(n-k)!
n(n-1)(n-2)…(n-k+1) (n-k)(n-k-1)…(1)(n-k)!
=
(n)k = n(n-1)(n-2)…(n-k+1)= n(n-1)(n-2)…(n-k+1)
Theorem: The number of k-permutations, (n)k , (ordered sequence) of n distinguishable objects is
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 35
Shuriken
Short Sword
Spear of Longinus
Gloves
Astral Spear
Example: You are allowed to choose only three from six items (in ordered sequence).
Death sickle
A
B
C
How many possible outcomes?
(n)k = n!(n-k)!
(6)3 =6!
(6-3)!6 x 5 x 4 x 3!
3!= = 120
Different
36
Theorem: Given n distinguishable objects, There are nk ways to choose with replacement a sample of k objects
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 37
Example: You are allowed to choose only three from six items (with replacement).
Shuriken
Short Sword
Spear of Longinus
Gloves
Astral Spear
Death sickle
B
How many possible outcomes?
A
C
nk = 6 x 6 x 6 = 216
38
Theorem: The number of ways to choose k objects out of n distinguishable objects is
( )nk =
(n)kk!
n!k!(n-k)!=
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
39
2 subexperiments: ( ) then (k)k
( ). (k)k = (n)k
nknk
63( ) = 20
No order
(3)3 = 6
Order(6)3 = 120Order
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
Shuriken
Short Sword
Spear of Longinus
Gloves
Astral Spear
Death sickle
1 2 3
40
• Perform repeated trials• p = a success probability• (1-p) = a failure probability• Each trial is independent• Sk,n = the event that k successes in n trials
( )nk
P[Sk,n] = pk(1-p)n-k
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
• 3 trials with 2 successes• 000 001 010 011 100 101 110 111• How many way to choose 2 out of 3
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 41
• What is the probability of success for each way ?• p2 * (1-p)
( )32
P[S2,3] = p2(1-p)3-2
( )nk= = = 3( )3
2
• Example: In the first round of a programming contest, probability that a program will pass the test is 0.8 .
• From 10 candidates, what is the probability that x candidates will pass?
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 42
10x( )P[Ax,10] = (0.8)x(1-0.8)10-x
P[A8,10] = (45)(0.1678)(0.04) = 0.3
Solution:A = {program pass the test}, P[A] = 0.8Testing a program is an independent trial
And what is P[x = 8]?
43
Let probability that a computer works = pSeries: P[A] = P[A1A2] = p2
Parallel: P[B] = ?
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty
P[B] = 1 – P[Bc] = 1 – P[B1
cB2c]
= 1 – (1 – p)2
ParallelSeries
• Probability meaning• Sample space, Event, Outcome• Set Theory• Probability measurement• Conditional Probability• Independence• Sequential experiments -> tree diagram• Counting Methods• Independent Trials
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 44