Training neural - Accueil - Département d'informatique …bengioy/cifar/NCAP2014-summer...Lower...
15
Training neural bayesian nets Laurent Dinh Vincent Dumoulin
Transcript of Training neural - Accueil - Département d'informatique …bengioy/cifar/NCAP2014-summer...Lower...
Training neural bayesian nets
Laurent Dinh!Vincent Dumoulin
Directed graphical model
❖ Fast and unbiased sampling!❖ Explaining away:
difficult inference!❖
Importance sampling
❖ Learning Stochastic Feedforward Neural Networks!
❖ Lower bound training
❖ Log likelihood > - 181.2
Variational autoencoder
❖ Variational boundFast approximate posterior sampling!
❖ Log likelihood: - 95
-
Variational autoencoder
Recognition Model
❖ Trained with high momentum!❖ Similar to Smoothed Gradient for Stochastic Variational Inference
Component collapsing
Depth
Depth
Effects on the model
Effects on the model
Depth
Depth
X X
Importance sampling with proposal distribution
❖ Use a learned proposal for the importance sampling!
❖ Lower bound training
❖ Marginally less collapsing!❖ Log likelihood: - 91.1
Thank you for your attention
![arXiv:cond-mat/9804274v1 [cond-mat.soft] 24 Apr 1998arXiv:cond-mat/9804274v1 [cond-mat.soft] 24 Apr 1998 Variational bound on energy dissipation in plane Couette flow Rolf Nicodemus∗,](https://static.fdocuments.in/doc/165x107/60f9196a05f06704071e95d4/arxivcond-mat9804274v1-cond-matsoft-24-apr-1998-arxivcond-mat9804274v1-cond-matsoft.jpg)
![Uncertainty in Bayesian Neural Nets · BNN •Variational Inference •Maximize lower bound on the marginal log-likelihood log’12 ≥!"$[log’12,,+log’,−log6,] Prior Posterior](https://static.fdocuments.in/doc/165x107/5fc2a276693fd84a34519fe5/uncertainty-in-bayesian-neural-nets-bnn-avariational-inference-amaximize-lower.jpg)
