Lectures 17,18 – Boosting and Additive Trees Rice ECE697 Farinaz Koushanfar Fall 2006.
ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and...
-
Upload
dora-newman -
Category
Documents
-
view
225 -
download
1
Transcript of ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and...
![Page 1: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/1.jpg)
ELEC 303, Koushanfar, Fall’09
ELEC 303 – Random Signals
Lecture 1 - General info, Sets and Probabilistic Models
Farinaz KoushanfarECE Dept., Rice University
Aug 25, 2009
![Page 2: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/2.jpg)
ELEC 303, Koushanfar, Fall’09
General information
• Syllabus/policy handout• Course webpage:
http://www.ece.rice.edu/~fk1/classes/ELEC303.htm
• Required textbook: Dimitri P. Bertsekas and John N. Tsitsiklis. Introduction to Probability. 2nd Edition, Athena Scientific Press, 2008
• Recommended books on webpage• Instructor and TA office hours on
the webpage
![Page 3: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/3.jpg)
ELEC 303, Koushanfar, Fall’09
Grading / policy
• Grading– Quiz 1: 10%– Midterm: 20%– Quiz 2: 10% – Final: 30% – Homework: 15%– Matlab assignments: 10%– Participation: 5%
• Read the policy handout for the homework policy, cheating policy, and other information of interest
![Page 4: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/4.jpg)
ELEC 303, Koushanfar, Fall’09
Lecture outline
• Reading: Sections 1.1, 1.2• Motivation• Sets• Probability models
– Sample space– Probability laws – axioms– Discrete and continuous models
![Page 5: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/5.jpg)
ELEC 303, Koushanfar, Fall’09
Motivation
• What are random signals and probability?• Can we avoid them?• Why are they useful?• What are going to learn?
![Page 6: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/6.jpg)
ELEC 303, Koushanfar, Fall’09
Sets – quick review
• A set (S) is a collection of objects (xi) which are the elements of S, shown by xiS, i=1,…,n
• S may be finite or countably infinite
![Page 7: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/7.jpg)
ELEC 303, Koushanfar, Fall’09
Sets – quick review
• Set operations and notations:– Universal set: , empty set: , complement: Sc
– Union:
– Intersection:
– De Morgan’s laws:
} some for |{...211
nSxxSSS nnn
} all for |{...211
nSxxSSS nnn
n
cn
c
nn SS )(
n n
cn
cn SS )(1 2
![Page 8: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/8.jpg)
ELEC 303, Koushanfar, Fall’09
Venn diagram
• A representation of sets
A Ac
A B
A B
A
B
C A
B
C
ST
U
![Page 9: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/9.jpg)
ELEC 303, Koushanfar, Fall’09
Probabilistic models
• Sample space: set of all possible outcomes of an experiment (mutually exclusive, collectively exhaustive)
• Probability law: assigns to a set A of possible outcomes (events) a nonnegative number P(A)– P(A) is called the probability of A
Figure courtesy of Bertsekas&Tsitsiklis, Introduction to Probability, 2008
![Page 10: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/10.jpg)
ELEC 303, Koushanfar, Fall’09
From frequency to probability (1)
Y,LO,L
0200400600800
1000120014001600
S,Fast U
Freq
uenc
yRecovery time
Age
Slide courtesy of Prof. Dahleh, MIT
The time of recovery (Fast, Slow, Unsuccessful) from an ACL knee surgery was seen to be a function of the patient’s age (Young, Old) and weight (Heavy, Light). The medical department at MIT collected the following data:
![Page 11: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/11.jpg)
ELEC 303, Koushanfar, Fall’09
From frequency to probability (2)
Y,LY,H O,
L O,H
0
400
800
1200
1600
S,Fast U
• What is the likelihood that a 40 years old man (old!) will have a successful surgery with a speedy recovery?
• If a patient undergoes operation, what is the likelihood that the result is unsuccessfull?
• Need a measure of “likelihood”• Ingredients: sample space, events, probability
Slide courtesy of Prof. Dahleh, MIT
![Page 12: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/12.jpg)
ELEC 303, Koushanfar, Fall’09
Sequential models – sample space
Slide courtesy of Prof. Dahleh, MIT
2 1
3
![Page 13: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/13.jpg)
ELEC 303, Koushanfar, Fall’09
Axioms of probability
1. (Nonnegativity) 0≤P(A)≤1 for every event A2. (Additivity) If A and B are two disjoint events,
then the probability P(AB)=P(A)+P(B)
3. (Normalization) The probability of the entire sample space is equal to 1, i.e., P()=1
![Page 14: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/14.jpg)
ELEC 303, Koushanfar, Fall’09
Discrete models
• Example: coin flip – head (H), tail (T)• Assume that it is a fair coin• What is the probability of getting a T?• What is the probability of getting 2 H’s in three
coin flips?• Discrete probability law for a finite number of
possible outcomes: the probability of an event is the sum of it’s disjoint elements’ probabilities
![Page 15: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/15.jpg)
ELEC 303, Koushanfar, Fall’09
Example: tetrahedral dice
• Let every possible outcome have probability 1/16
• P(X=1)=P(X=2)=P(X=3)=P(X=4)=0.25• Define Z=min(X,Y)• P(Z=1)=?• P(Z=2)=?• P(Z=3)=?• P(Z=4)=?
2 1
3
Discrete uniform law:Let all sample points be equally likely, then
points sample ofnumber total
A of elements ofnumber )( AP
Example courtesy of Prof. Dahleh, MIT
![Page 16: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/16.jpg)
ELEC 303, Koushanfar, Fall’09
Continuous probability
• Each of the two players randomly chose a number in [0,1]. What is the probability that the two numbers are at most ¼ apart?
• Draw the sample space and event area• Choose a probability law
– Uniform law: probability = area
¼¼ x
y
1
1
Example courtesy of Prof. Dahleh, MIT
![Page 17: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/17.jpg)
ELEC 303, Koushanfar, Fall’09
Some properties of probability laws
• If A B, then P(A) ≤ P(B)• P(AB) = P(A) + P(B) - P(AB)• P(AB) ≤ P(A) + P(B)• P(AB C) = P(A) + P(AcB) + P(AcBcC)
![Page 18: ELEC 303, Koushanfar, Fall’09 ELEC 303 – Random Signals Lecture 1 - General info, Sets and Probabilistic Models Farinaz Koushanfar ECE Dept., Rice University.](https://reader031.fdocuments.in/reader031/viewer/2022032203/56649e175503460f94b01efa/html5/thumbnails/18.jpg)
ELEC 303, Koushanfar, Fall’09
Models and reality
• Probability is used to model uncertainty in real world
• There are two distinct stages:– Specify a probability law suitably defining the
sample space. No hard rules other than the axioms– Work within a fully specified probabilistic model
and derive the probabilities of certain events