ENSC327 Communications Systems 16: Probability (Chap. 8)
Transcript of ENSC327 Communications Systems 16: Probability (Chap. 8)
ENSC327
Communications Systems
16: Probability (Chap. 8)
1
Jie Liang
School of Engineering Science
Simon Fraser University
Probability
� Most signals of interest are random.
� The performance of communication systems is affected by random noise.
� Probability theory and random process are needed.
� History of Probability Theory:
2
� History of Probability Theory:� Created in 1654 by Pascal and Fermat after a question from a gambler.
� 1812: Laplace applied probabilistic ideas to many scientific and practical problems.
� Several definitions have been developed.
� 1933: Kolmogorov outlined an axiomatic definition of probability that formed the basis of the modern theory.
� Probability theory is now part of a more general discipline known as measure theory.
1 Classical Definition of Probability
� Also called a priori Definition of Probability� Allows probabilities to be computed in special cases without
experimentation.
� Most notably, probabilities can be computed in games of chance.
� A priori probabilities are most commonly computed for equally likely outcomes
3
outcomes
� A random experiment is performed.
� Outcome: The result of a random experiment.
� Event: a collection of outcomes.
� The probability of event A is defined as:
� n: Total number of outcomes
� nA: Number of favorable outcomes belonging to an event A.
n
nAP
A=][
1 Classical Definition of Probability
� If we randomly draw a card from a deck of 52 cards:
� Number of possible outcomes:
� If the event of interest is drawing a Heart:
4
� If the event of interest is drawing a King:
� If the event of interest is drawing a King OR a Heart:
2. Relative Frequency Definition of
Probability
� The Relative Frequency approach to probability is
� Limitation of the classical definition: it implicitly
defines all outcomes to be equiprobable, which is not
always true.
5
� The Relative Frequency approach to probability is
well-suited to a wide range of scientific disciplines.
� Assumption: the probability of an event can be
measured by repeated trials.
� The probability of event A is defined as
n
nAP
A
n
lim][∞→
=
Some Notations from Set Theory
�Union of A and B:
� Given two sets A and B, their union is the set
consisting of all objects which are elements of A
or of B or of both.
BA∪
6
or of B or of both.
�Intersection of A and B:
�The intersection of A and B is the set of all objects
which are both in A and in B.
Some Notations from Set Theory
�Complement of A:
7
�Null Set: null or impossible event
3 Axiomatic Definition of Probability
� Most general approach to probability.
� Treat a random experiment and its outcomes as a sample space S and its points.
� Each possible outcome is mapped to a sample point sk.
� An event corresponds to a single sample point or a set
8
� An event corresponds to a single sample point or a set of sample points.
� A single sample point is called an elementary event.
� The entire sample space S is called the sure event.
Axiomatic Definition of Probability
� A probability system consists of the triple:
1. A sample space S of elementary events (outcomes).
2. A class of events that are subsets of S.
3. A probability measure P[A] to each event A with the following axims:
9
following axims:
(i) P[S] = 1.
(ii) 0 ≤ P[A] ≤ 1.
(iii)
exclusive.mutually are
B andA if )()()( BPAPBAP +=∪
The Venn Diagrams
�Event: a collection of outcomes.
10
sample space, S
Some Properties of ProbabilityAxims:
(i) P[S] = 1.
(ii) 0 ≤ P[A] ≤ 1.
(iii) B. andA exclusivemutually for )()()( BPAPBAP +=∪
11
1 ( )( ) P AP A = -Property 1:
Some Properties of Probability
Property 2:
Proof:
).(- )()()( BAPBPAPBAP ∩+=∪
12
Some Properties of Probability
� Joint probability:
� Prob. that both event A and B occur
Marginal Probability:
13
� Marginal Probability:
� The probability of one event, regardless of the other event.
� Obtained by summing the joint probability over the un-
required event.
Some Properties of Probability
�Conditional Probability of event A given event
B occurred:
14
�Conditional Probability of B given A:
Some Properties of Probability
�Bayes’ rule:
15
Bayes’ rule is useful when P(A|B), P(A) and P(B) can be
easily determined, but P(B|A) is desired.
Some Properties of Probability
� Independence: Events A and B are
independent if
( )( | )
or ( | ) ( )
P A B P A
P B A P B
=
=
16
or ( | ) ( )P B A P B=
Law of Total Probability
� If {B1, B2, …, Bn} are pairwise disjoint and
their union is the entire sample space, then
for any event A of the same sample space:
17
Example
� Tossing two fair coins simultaneously:
� Possible outcomes: HH, HT, TH, TT
� Event A: at least one head (HH, HT, TH)
� Event B: a match (HH, TT)
18
( ) ?P A = ( ) ?P B =
Are A and B independent?
Example
�Consider a Binary Symmetric Channel (BSC)
s = r =
P(0r | 0s) = 1 -
P(1r | 1s) = 1 -
P(1r | 0s) =
ε
ε
ε
19
�Questions:
� Prob. of getting j errors in k bits?
� Most probable input given that a “1” is received?
P(1r | 0s) =
P(0r | 1s) =
ε
ε
P(0s) = p
1st Question
�The prob. of j errors in k bits:
� Assume the transmissions of different bits are
independent
20
2nd Question
�Most probable input given we receive a “1”?
� Need to compare
�The most probable input is 1 if
21
�The most probable input is 1 if
�The most probable input is 0 if
��Maximum a-posteriori (MAP) criterion
2nd Question
22
Example
�Assume p = 0.8, 1.0=ε
23