Unit1_ Lec 1_ ITA

Upload
ayushgarg 
Category
Documents

view
266 
download
2
Transcript of Unit1_ Lec 1_ ITA

7/24/2019 Unit1_ Lec 1_ ITA
1/32
Review of Basic Probability
Unit1

7/24/2019 Unit1_ Lec 1_ ITA
2/32
What is Probability?
Deterministic phenomena
Daily sunrises andsunsets
Tides at sea shores
Phases of the moon
Seasonal changes in
weather
Annual flooding of theYamuna
Random phenomena
Results of coin tosses
Results of rolling dice
Results of horse races
World refer to dice games
and gambling

7/24/2019 Unit1_ Lec 1_ ITA
3/32
Probabilistic notions are common placein everyday language usage
We use words such as
Probable/improbable; possible/impossible
Certain/uncertain; likely/unlikely
Phrases such as there is a 5050 chance
The probability of precipitation is 20%

7/24/2019 Unit1_ Lec 1_ ITA
4/32
PROBABILITY ?
The theory of probability deals with averagesof mass phenomenon occurring sequentially orsimultaneously.
The purpose of theory is to describe andpredict averages in terms of probabilities ofevents.

7/24/2019 Unit1_ Lec 1_ ITA
5/32
Definition
Three Aproaches to Probability:
Classical
Relative frequency Axiomatic

7/24/2019 Unit1_ Lec 1_ ITA
6/32
Classical (Priori) Approach to Probability:
The probability of Heads is because there are twosides to the coin
This is called the classical approach to probability
More generally, if there are n possible outcomes of anexperiment, then each outcome has probability 1/n
Justification: Symmetry principle (Equally likely);

7/24/2019 Unit1_ Lec 1_ ITA
7/32
Problems with Classical Approach:
What exactly is an outcome?If we toss two coins, are there three outcomes or fouroutcomes?
{0 Heads, 1Head, 2 Heads}?
{(T,T), ( T,H), ( H,T), ( H,H)}?
Note that 2 Heads has probability 1/3 or dependingon the choice
There are only two outcomes: either I Win theLottery, or I dont, so probability is 1/2?

7/24/2019 Unit1_ Lec 1_ ITA
8/32
Relative Frequency (Posteriori) Approach:
the probability of Heads is because when tossed, thecoin will turn up Heads half the time
How do we know the coin will turn up Heads half the
time? Suppose multiple heads tosses have resulted in 50%
Heads.
Setting P (Head)=1/2 is the relative frequency approach
to probability

7/24/2019 Unit1_ Lec 1_ ITA
9/32

7/24/2019 Unit1_ Lec 1_ ITA
10/32
Relative Frequency (Posteriori)Approach contd
If an outcome x occurs m times on N trails, its
relative frequency is m/N & we define itsprobability P(x) to be m/N
Does there exist a probability of Heads for newunbiased untossed coin?
Or do probabilities come into existence only aftermultiple tosses?
How large N should be?
Are probabilities redefined after each toss?

7/24/2019 Unit1_ Lec 1_ ITA
11/32
Probability as beliefs:
Many assertions about probability are
essentially statements of beliefs A fair coin is one for which P(Heads)=1/2
but how do we know whether a given coin
is fair? Symmetry of the physical object is a belief
That further tosses of a coin for which
P(Heads) =1/2 will result in 50% Heads isa belief

7/24/2019 Unit1_ Lec 1_ ITA
12/32
Axiomatic Approach to Probability:
In the axiomatic approach, probabilities are numbers
in the range [0, 1] Certain probabilities are assumed to be given (we
dont ask how!)
allows the calculations of other probabilities in amathematically & logically consistent manner
It is probability calculas
allows the computation of probabilities withoutrequiring philosophical discussions about the meaningof probability
Consistent with all the approaches described above

7/24/2019 Unit1_ Lec 1_ ITA
13/32
So , t h e P r o b a b i l i t y c a n b e d e f i n e d
a s t h e M e a s u r e o f t h e p o s s i b i l i t i e s o f
o c cu r r e n c e o f a n e v e n t i n a
r a n d o m e x p e r im e n t .
P r o b a b i l i t y o f a n e v e n t A i s
d e n o t e d b y P ( A ) .

7/24/2019 Unit1_ Lec 1_ ITA
14/32
Probability in Engineering:
Thermal noise in electrical circuits
Information Theory
Communication systems design
Noise
Games of Chance
Reliability of systems
Failure probabilities
Failure Rates
Mean time to failure

7/24/2019 Unit1_ Lec 1_ ITA
15/32
Networks & Systems Problems
Random arrivals of packets/jobs
Random lengths/service times
Random requests for resources
Probability of buffer or queue overflow
Transmission or service delays
Scheduling problem, priorities, QOS
Flow control and routing

7/24/2019 Unit1_ Lec 1_ ITA
16/32
Trial ?
Experiement ?
Outcome ?
Sample space ? Event ?

7/24/2019 Unit1_ Lec 1_ ITA
17/32
Experiments and Trials
Fundamental notion: An experiment is
performed and its outcome observed
This is called a trail of experiment
The experiment may be performed by a
human agent, e.g., tossing a coin
or rolling a dice
The experimental outcome might just be
the measurement of a naturally occurringrandom phenomenon, e.g. a noise voltage

7/24/2019 Unit1_ Lec 1_ ITA
18/32
The Sample Space
The set of all possible outcomes of anexperiment is called sample space of the experiment
Examples: The experiment isTossing a coin: ={H,T}
rolling a dice: ={1,2,3,4,5,6}
noise voltage: ={x:1x1}

7/24/2019 Unit1_ Lec 1_ ITA
19/32
Exercise
Example: The experiment is
rolling a dice: ={1,2,3,4,5,6}
suppose that each outcome is equally likely:
P(1)= P(2)= P(3)= P(4)= P(5)= P(6)
What is probability of rolling an evennumber?
What is probability of rolling an prime
number?

7/24/2019 Unit1_ Lec 1_ ITA
20/32
An even number is said to have beenrolled if the outcome is any of {2,4,6}
P (even number)=1/2; more explicitly
P (even number)=3/6 since 3 of the 6outcomes are in the subset {2,4,6}
An prime number is said to have been
rolled if the outcome is any of {2,3,5} P (prime number)=1/2 also

7/24/2019 Unit1_ Lec 1_ ITA
21/32
Event
A subset of is called an event
Example: A={2,4,6} & B={2,3,5}are said to be events defined onsample space ={1,2,3,4,5,6}
events defined on the sample spaceis merely a probabilists way of sayingsubsets of the sample space
Ac={1,3,5} & Bc={1,4,6} also areevents define on

7/24/2019 Unit1_ Lec 1_ ITA
22/32
When does an event occur?
An event A is said to have occurred on a trial
if the outcome of the trial is a member of thesubset A
Event A occurs if the observed outcome is
some member of A; we dont care whichmember of A it is
If the observed outcome is not a member ofA, then we say A did not occur, orequivalently, we say that Ac occurred

7/24/2019 Unit1_ Lec 1_ ITA
23/32
Outcomes vs. Events
Every trial results in only one outcome, i.e.,
only one of the elements in can be observedoutcome
The observed outcome is a member of several
different subsets, i.e., events & all theseevents are said to have occurred
Fundamental notion: on each trial of the
experiment, one outcomes occurs, butmany events occur

7/24/2019 Unit1_ Lec 1_ ITA
24/32
Example: If outcome of rolling adice is 4, then
Events A ={2,4,6} & Bc={1,4,6} both
have occurred
Events Ac={1,3,5} & B={2,3,5} did
not occur Event A U Bc = {1,2,4,6} has
occurred
Event A and Bc = {4,6} has occurred

7/24/2019 Unit1_ Lec 1_ ITA
25/32
Two special events
can be regarded as a subset of
On any trial, the event always occurs The event is called the certain event or the
sure event
, the empty set, is also a subset of On any trial, the event never occurs
The event is called null event or the impossibleevent
A sample space of n elements has 2n
different subsets including &

7/24/2019 Unit1_ Lec 1_ ITA
26/32
Probabilities of the special events
always occurs; c= never occurs
Conclusion: the probabilities assignedto & should be 1 & 0respectively, regardless of how wechoose to assign probabilities to theoutcomes
P()=1 will be used as an axiom inthe axiomatic approach to probability

7/24/2019 Unit1_ Lec 1_ ITA
27/32
Arbitrary probability assignment
Nonclassical approach: The n outcomes
have probabilities p1, p2 ...pn where pi0& pi =1
The probability of an event A is the sum
of the probabilities of all the outcomesthat comprise A
P(A)=sum of the pi for all members of A
Example: A={x2,x4,x22}P(A)=p2+p4+p22

7/24/2019 Unit1_ Lec 1_ ITA
28/32
Disjoint Events
Events A & B are said to be disjoint or
mutually exclusive if A & B have no element in common
A ={1,3,5} & B={2,4,6} are disjoint events
A U B = {1,2,3,4,5,6}
P(A U B ) =p1+p2+p3+p4+p5+p6= P(A)+P(B)
=I BA

7/24/2019 Unit1_ Lec 1_ ITA
29/32
Probability Axioms for finite spaces
Probabilities are numbers assigned to events
that satisfy the following rules: Axiom I: P(A)0 for all events A
Axiom II: P()=1
Axiom III: If events A & B are disjoint, thenP(A U B ) = P(A)+P(B)
Consequences: P()=0
P(Ac)=1  P(A); P(A)=1  P(Ac)
0P(A)1 for all events A

7/24/2019 Unit1_ Lec 1_ ITA
30/32
Countably infinite sample spaces
Let = {x1,x2,.xn,.} be the Countably
infinite sample spaces P{xn}=pn where pn0
For a finite subset A of , P(A) is just the
sum of the probabilities of the outcomescomprising the event A, as before
It seems reasonable to have this idea work
for an infinite subset of A as well But we need a new improved axiom

7/24/2019 Unit1_ Lec 1_ ITA
31/32
New improved Axiom III
Let A1,A2,..An denote a countable sequence
of disjoint events, i.e.,for all ij. Then, P(A1UA2U.. UAnU. )
=P(A1)+ P(A1)+ P(A1)+..P(An)+.
The new axiom implies that P() =0 &
P(A U B ) = P(A)+P(B) for AB =
=I ji AA

7/24/2019 Unit1_ Lec 1_ ITA
32/32
The Probability Space:
Formal statement of the axiomatic
theory A probability space (,F,P) consists of
The sample space consisting of all
possible outcomes of the experiment The field of events F which includes
all the interesting subsets of
The probability measure P() thatassigns probabilities to the events