An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum...
-
Upload
melanie-hubbard -
Category
Documents
-
view
216 -
download
1
Transcript of An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum...
![Page 1: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/1.jpg)
An Efficient Approach to Learning Inhomogenous Gibbs Models
Ziqiang Liu, Hong Chen, Heung-Yeung ShumMicrosoft Research AsiaCVPR 2003
Presented by Derek Hoiem
![Page 2: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/2.jpg)
Overview
Build histograms for projections to 1-D Feature selection: max KL divergence between
estimated and true distribution 1-D histograms for a feature computed from
training data and MCMC sampling Fast solution with good starting point and
importance sampling
![Page 3: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/3.jpg)
Maximum Entropy Principle
p(x) and f(x) should have same stats over observed features but p(x) should be as random as possible over other dimensions
![Page 4: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/4.jpg)
Gibbs Distribution and KL-Divergence
The solution: Gibbs distribution
Λ minimizes the KL divergence:
![Page 5: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/5.jpg)
Inhomogeneous Gibbs Model
Gaussian and MoG deemed inadequate Use vector-valued features (histograms)
![Page 6: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/6.jpg)
Approximate Information Gain and KL-Divergence
Effectiveness of feature defined by reduction in KL-divergence:
Approximate information gain given by (old params constant):
For a vector-valued feature: KeyContribution!
gain starting point
![Page 7: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/7.jpg)
Estimating Λ: Importance Sampling
Obtain reference samples xref by MCMC from starting point Update Λ by:
Bad starting point
Good starting point
![Page 8: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/8.jpg)
A Toy Success Story
True
Reference (Initial)
Optimized Estimate
![Page 9: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/9.jpg)
Caricature Generation: Representation
Learn mapping from photo to caricature Active appearance models:
Photos: shape + texture (44-D after PCA)Caricature: shape (25-D after PCA)
![Page 10: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/10.jpg)
Caricature Generation: Learning
Gain(1)=.447 Gain(17)=.196 100,000 reference samples 8 hours on 1.4GHz 256MB
vs 24 hours on 667MHz 18-D
Estimate: Draw samples from: Approximate to:
![Page 11: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/11.jpg)
Caricature Generation: Results
![Page 12: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/12.jpg)
Caricature Generation: Results
![Page 13: An Efficient Approach to Learning Inhomogenous Gibbs Models Ziqiang Liu, Hong Chen, Heung-Yeung Shum Microsoft Research Asia CVPR 2003 Presented by Derek.](https://reader036.fdocuments.in/reader036/viewer/2022062719/56649ee45503460f94bf3a4f/html5/thumbnails/13.jpg)
Comments
Claims 100x speedup from efficiency analysis (33% speedup in reality)