Book - Introduction to Probability (2e) by Dimitri P. Bertsekas & John N. Tsitsiklis
EEL 5544 - Noise in Linear Systems (Should be called...
Transcript of EEL 5544 - Noise in Linear Systems (Should be called...
![Page 1: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/1.jpg)
EEL 5544 - Noise in LinearSystems
(Should be called ldquoProbability andRandom Processes for Electrical
Engineersrdquo)
Dr John M Shea
Fall 2008
EEL5544
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required
EEL5544 S-1
Meeting Time 935ndash1025 MondayWednesdayFriday
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 2: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/2.jpg)
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required
EEL5544 S-1
Meeting Time 935ndash1025 MondayWednesdayFriday
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 3: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/3.jpg)
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required
EEL5544 S-1
Meeting Time 935ndash1025 MondayWednesdayFriday
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 4: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/4.jpg)
Meeting Time 935ndash1025 MondayWednesdayFriday
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 5: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/5.jpg)
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 6: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/6.jpg)
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 7: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/7.jpg)
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 8: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/8.jpg)
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 9: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/9.jpg)
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 10: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/10.jpg)
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 11: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/11.jpg)
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 12: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/12.jpg)
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 13: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/13.jpg)
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 14: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/14.jpg)
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 15: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/15.jpg)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 16: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/16.jpg)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 17: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/17.jpg)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 18: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/18.jpg)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 19: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/19.jpg)
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 20: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/20.jpg)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 21: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/21.jpg)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 22: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/22.jpg)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 23: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/23.jpg)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 24: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/24.jpg)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 25: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/25.jpg)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 26: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/26.jpg)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 27: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/27.jpg)
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 28: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/28.jpg)
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 29: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/29.jpg)
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 30: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/30.jpg)
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 31: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/31.jpg)
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 32: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/32.jpg)
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 33: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/33.jpg)
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 34: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/34.jpg)
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 35: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/35.jpg)
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 36: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/36.jpg)
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 37: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/37.jpg)
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 38: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/38.jpg)
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 39: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/39.jpg)
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 40: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/40.jpg)
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 41: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/41.jpg)
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 42: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/42.jpg)
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 43: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/43.jpg)
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 44: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/44.jpg)
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 45: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/45.jpg)
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 46: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/46.jpg)
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 47: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/47.jpg)
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
![Page 48: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/48.jpg)
![Page 49: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/49.jpg)
![Page 50: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/50.jpg)
![Page 51: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/51.jpg)
![Page 52: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/52.jpg)
![Page 53: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/53.jpg)
![Page 54: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/54.jpg)
![Page 55: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/55.jpg)
![Page 56: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/56.jpg)
![Page 57: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/57.jpg)
![Page 58: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/58.jpg)
![Page 59: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/59.jpg)
![Page 60: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/60.jpg)
![Page 61: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/61.jpg)
![Page 62: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/62.jpg)
![Page 63: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/63.jpg)
![Page 64: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/64.jpg)
![Page 65: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/65.jpg)
![Page 66: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/66.jpg)
![Page 67: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/67.jpg)
![Page 68: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/68.jpg)
![Page 69: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/69.jpg)
![Page 70: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/70.jpg)
![Page 71: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/71.jpg)
![Page 72: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/72.jpg)
![Page 73: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/73.jpg)
![Page 74: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/74.jpg)
![Page 75: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/75.jpg)
![Page 76: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/76.jpg)
![Page 77: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/77.jpg)
![Page 78: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/78.jpg)
![Page 79: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/79.jpg)
![Page 80: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/80.jpg)
![Page 81: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/81.jpg)
![Page 82: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/82.jpg)
![Page 83: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/83.jpg)
![Page 84: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/84.jpg)
![Page 85: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/85.jpg)
![Page 86: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/86.jpg)
![Page 87: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/87.jpg)
![Page 88: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/88.jpg)
![Page 89: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/89.jpg)
![Page 90: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/90.jpg)
![Page 91: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/91.jpg)
![Page 92: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/92.jpg)
![Page 93: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/93.jpg)
![Page 94: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/94.jpg)
![Page 95: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/95.jpg)
![Page 96: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/96.jpg)
![Page 97: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/97.jpg)
![Page 98: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/98.jpg)
![Page 99: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/99.jpg)
![Page 100: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/100.jpg)
![Page 101: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/101.jpg)
![Page 102: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/102.jpg)
![Page 103: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/103.jpg)
![Page 104: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/104.jpg)
![Page 105: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/105.jpg)
![Page 106: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/106.jpg)
![Page 107: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/107.jpg)
![Page 108: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/108.jpg)
![Page 109: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/109.jpg)
![Page 110: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/110.jpg)
![Page 111: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/111.jpg)
![Page 112: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/112.jpg)
![Page 113: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/113.jpg)
![Page 114: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/114.jpg)
![Page 115: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/115.jpg)
![Page 116: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/116.jpg)
![Page 117: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/117.jpg)
![Page 118: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/118.jpg)
![Page 119: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/119.jpg)
![Page 120: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/120.jpg)
![Page 121: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/121.jpg)
![Page 122: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/122.jpg)
![Page 123: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/123.jpg)
![Page 124: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/124.jpg)
![Page 125: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/125.jpg)
![Page 126: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/126.jpg)
![Page 127: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/127.jpg)
![Page 128: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/128.jpg)
![Page 129: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/129.jpg)
![Page 130: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/130.jpg)
![Page 131: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/131.jpg)
![Page 132: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/132.jpg)
![Page 133: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/133.jpg)
![Page 134: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/134.jpg)
![Page 135: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/135.jpg)
![Page 136: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/136.jpg)
![Page 137: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/137.jpg)
![Page 138: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/138.jpg)
![Page 139: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/139.jpg)
![Page 140: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/140.jpg)
![Page 141: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/141.jpg)
![Page 142: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/142.jpg)
![Page 143: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/143.jpg)
![Page 144: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/144.jpg)
![Page 145: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/145.jpg)
![Page 146: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/146.jpg)
![Page 147: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/147.jpg)
![Page 148: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/148.jpg)
![Page 149: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/149.jpg)
![Page 150: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/150.jpg)
![Page 151: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/151.jpg)
![Page 152: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/152.jpg)
![Page 153: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/153.jpg)
![Page 154: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/154.jpg)
![Page 155: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/155.jpg)
![Page 156: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/156.jpg)
![Page 157: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/157.jpg)
![Page 158: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/158.jpg)
![Page 159: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/159.jpg)
![Page 160: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/160.jpg)
![Page 161: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/161.jpg)
![Page 162: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/162.jpg)
![Page 163: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/163.jpg)
![Page 164: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/164.jpg)
![Page 165: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/165.jpg)
![Page 166: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/166.jpg)
![Page 167: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/167.jpg)
![Page 168: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/168.jpg)
![Page 169: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/169.jpg)
![Page 170: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/170.jpg)
![Page 171: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/171.jpg)
![Page 172: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/172.jpg)
![Page 173: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/173.jpg)
![Page 174: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/174.jpg)
![Page 175: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/175.jpg)
![Page 176: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/176.jpg)
![Page 177: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/177.jpg)
![Page 178: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/178.jpg)
![Page 179: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/179.jpg)
![Page 180: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/180.jpg)
![Page 181: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/181.jpg)
![Page 182: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/182.jpg)
![Page 183: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/183.jpg)
![Page 184: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/184.jpg)
![Page 185: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/185.jpg)
![Page 186: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/186.jpg)
![Page 187: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/187.jpg)
![Page 188: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/188.jpg)
![Page 189: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/189.jpg)
![Page 190: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/190.jpg)
![Page 191: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/191.jpg)
![Page 192: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/192.jpg)
![Page 193: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/193.jpg)
![Page 194: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/194.jpg)
![Page 195: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/195.jpg)
![Page 196: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/196.jpg)
![Page 197: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/197.jpg)
![Page 198: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/198.jpg)
![Page 199: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/199.jpg)
![Page 200: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/200.jpg)
![Page 201: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/201.jpg)
![Page 202: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/202.jpg)
![Page 203: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/203.jpg)
![Page 204: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/204.jpg)
![Page 205: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/205.jpg)
![Page 206: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/206.jpg)
![Page 207: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/207.jpg)
![Page 208: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/208.jpg)
![Page 209: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/209.jpg)
![Page 210: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/210.jpg)
![Page 211: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/211.jpg)
![Page 212: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/212.jpg)
![Page 213: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/213.jpg)
![Page 214: EEL 5544 - Noise in Linear Systems (Should be called ...wireless.ece.ufl.edu/eel5544/lectures/lecture1_overheads1.pdfprobability, I suggest: D. P. Bertsekas and J. N. Tsitsiklis, Introduction](https://reader035.fdocuments.in/reader035/viewer/2022071406/60fbd5a02dfe20485174a7b9/html5/thumbnails/214.jpg)