Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2...
-
date post
22-Dec-2015 -
Category
Documents
-
view
217 -
download
1
Transcript of Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2...
![Page 1: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/1.jpg)
Bayesian Learning,cont’d
![Page 2: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/2.jpg)
Administrivia•Homework 1 returned today (details in a
second)
•Reading 2 assigned today
•S. Thrun, Learning occupancy grids with forward sensor models. Autonomous Robots, 2002.
•Due: Oct 26
•Much crunchier than the first! Don’t slack.
•Work with your group to sort out the math.
•Questions to mailing list and me.
•Midterm exam: Oct 21
![Page 3: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/3.jpg)
Homework 1 results•Mean=30.3; std=6.9
![Page 4: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/4.jpg)
IID Samples•In supervised learning, we usually
assume that data points are sampled independently and from the same distribution
•IID assumption: data are independent and identically distributed
•⇒ joint PDF can be written as product of individual (marginal) PDFs:
![Page 5: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/5.jpg)
The max likelihood recipe•Start with IID data
•Assume model for individual data point, f(X;Θ)
•Construct joint likelihood function (PDF):
•Find the params Θ that maximize L
•(If you’re lucky): Differentiate L w.r.t. Θ, set =0 and solve
•Repeat for each class
![Page 6: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/6.jpg)
Exercise•Find the maximum likelihood estimator of μ
for the univariate Gaussian:
•Find the maximum likelihood estimator of β for the degenerate gamma distribution:
•Hint: consider the log of the likelihood fns in both cases
![Page 7: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/7.jpg)
Solutions•PDF for one data point:
•Joint likelihood of N data points:
![Page 8: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/8.jpg)
Solutions•Log-likelihood:
![Page 9: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/9.jpg)
Solutions•Log-likelihood:
•Differentiate w.r.t. μ:
![Page 10: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/10.jpg)
Solutions•Log-likelihood:
•Differentiate w.r.t. μ:
![Page 11: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/11.jpg)
Solutions•Log-likelihood:
•Differentiate w.r.t. μ:
![Page 12: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/12.jpg)
Solutions•Log-likelihood:
•Differentiate w.r.t. μ:
![Page 13: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/13.jpg)
Solutions•What about for the gamma PDF?
![Page 14: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/14.jpg)
Putting the parts together
[X,Y]
com
ple
te
train
ing
data
![Page 15: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/15.jpg)
Putting the parts together Assumed distribution
family (hyp. space)w/ parameters Θ
Parameters for class a:
Specific PDFfor class a
![Page 16: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/16.jpg)
Putting the parts together
![Page 17: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/17.jpg)
Putting the parts together
![Page 18: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/18.jpg)
Gaussian Distributions
![Page 19: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/19.jpg)
5 minutes of math...•Recall your friend the Gaussian PDF:
•I asserted that the d-dimensional form is:
•Let’s look at the parts...
![Page 20: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/20.jpg)
5 minutes of math...
![Page 21: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/21.jpg)
5 minutes of math...•Ok, but what do the parts mean?
•Mean vector, : mean of data along each dimension
![Page 22: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/22.jpg)
5 minutes of math...•Covariance matrix
•Like variance, but describes spread of data
![Page 23: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/23.jpg)
5 minutes of math...•Note: covariances on the diagonal of
are same as standard variances on that dimension of data
•But what about skewed data?
![Page 24: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/24.jpg)
5 minutes of math...•Off-diagonal covariances ( )
describe the pairwise variance
•How much xi changes as x
j changes (on
avg)
![Page 25: Bayesian Learning, cont’d. Administrivia Homework 1 returned today (details in a second) Reading 2 assigned today S. Thrun, Learning occupancy grids with.](https://reader036.fdocuments.in/reader036/viewer/2022062407/56649d805503460f94a63a1f/html5/thumbnails/25.jpg)
5 minutes of math...•Calculating from data:
•In practice: you want to measure the covariance between every pair of random variables (dimensions):
•Or, in linear algebra: