Variational Inference

13
Part 2: Scalable Approximate Inference Session 1: - Variational Inference Session 2: - Sampling methods Approximate and Scalable Inference for Complex Probabilistic Models in Recommender Systems

Transcript of Variational Inference

Page 1: Variational Inference

Part 2: Scalable Approximate Inference

Session 1:- Variational Inference

Session 2:- Sampling methods

Approximate and Scalable Inference for Complex Probabilistic Models in Recommender Systems

Page 2: Variational Inference

Introduction

Page 3: Variational Inference

Motivation: bayesian mixture model

Page 4: Variational Inference

Main idea

Page 5: Variational Inference

KL-Divergence

KL of the posterior

Page 6: Variational Inference

KL-Divergence

Page 7: Variational Inference

ELBO e KL-Divergence

Jensen inequality concave (log)

KL of the posterior

Evidence lower bound (ELBO)

+

+

Page 8: Variational Inference

KL and ELBO

Choose family of variational distributions such that the expectations of log(q(z)) and log(p(x,z)) are computable

Page 9: Variational Inference

Mean-field

Page 10: Variational Inference

Optimizing a functional

Euler-lagrange equation

Page 11: Variational Inference

Mean-field

Page 12: Variational Inference
Page 13: Variational Inference

Structured variational inference


Variational Learning and Variational Inference · The variational approach • Variational inference: Find q(h) by solving • Variational learning: Alternate between running variational

Variational Learning and Variational Inference · The variational approach • Variational inference: Find q(h) by solving • Variational learning: Alternate between running variational

Variational Inference - Marc DeisenrothVariational Inference Variational inference is the mostscalable inference method available (at the moment) Can handle (arbitrarily) large datasets

Variational Inference - Marc DeisenrothVariational Inference Variational inference is the mostscalable inference method available (at the moment) Can handle (arbitrarily) large datasets

Functional Variational Bayesian Neural Networksbayesiandeeplearning.org/2018/papers/12.pdf · Sampling-Based Functional Variational Inference Adversarial functional variational inference

Functional Variational Bayesian Neural Networksbayesiandeeplearning.org/2018/papers/12.pdf · Sampling-Based Functional Variational Inference Adversarial functional variational inference

Tightening Bounds for Variational Inference by Revisiting ......Tightening Bounds for Variational Inference by Revisiting Perturbation Theory 4 Exact posterior inference is impossible

Tightening Bounds for Variational Inference by Revisiting ......Tightening Bounds for Variational Inference by Revisiting Perturbation Theory 4 Exact posterior inference is impossible

Stochastic Variational Inference - Columbia Universityjwp2128/Papers/HoffmanBleiWangPaisley2013.pdf · Using stochastic variational inference, we analyze several large collections

Stochastic Variational Inference - Columbia Universityjwp2128/Papers/HoffmanBleiWangPaisley2013.pdf · Using stochastic variational inference, we analyze several large collections

Variational Inference: A Review for Statisticians · inference (Hoffman et al., 2013), which scales variational inference to massive data using stochastic optimization (Robbins and

Variational Inference: A Review for Statisticians · inference (Hoffman et al., 2013), which scales variational inference to massive data using stochastic optimization (Robbins and

Variational Inference

Variational Inference

Alpha-Beta Divergence For Variational Inference

Alpha-Beta Divergence For Variational Inference

Variational Inference for Structured NLP Models

Variational Inference for Structured NLP Models

Morphogenesis as Bayesian Inference: A Variational ...

Morphogenesis as Bayesian Inference: A Variational ...

Generative Particle Variational Inference via Estimation ...

Generative Particle Variational Inference via Estimation ...

Variational Inference and Mean Field · Variational Inference “Variational inference”: Formulate inference problem as constrained optimization. Approximate the function or constraintsto

Variational Inference and Mean Field · Variational Inference “Variational inference”: Formulate inference problem as constrained optimization. Approximate the function or constraintsto

Variational Inference for Crowdsourcingpapers.nips.cc/paper/4627-variational-inference-for-crowdsourcing.pdf · variational inference methods for graphical models. First, we present

Variational Inference for Crowdsourcingpapers.nips.cc/paper/4627-variational-inference-for-crowdsourcing.pdf · variational inference methods for graphical models. First, we present

BIDIRECTIONAL VARIATIONAL INFERENCE FOR NON …

BIDIRECTIONAL VARIATIONAL INFERENCE FOR NON …

Variational Bayesian inference - Kay Brodersen · Variational Bayesian inference is based on variational calculus. Variational calculus Standard calculus Newton, Leibniz, and others

Variational Bayesian inference - Kay Brodersen · Variational Bayesian inference is based on variational calculus. Variational calculus Standard calculus Newton, Leibniz, and others

Variational Inference for LDAlzhang/teach/6931a/slides/lda-zhou.pdf4 Variational Inference E-Step M-Step 5 Conclusion. Generative Process of LDAExponential FamilyNewton Method Variational

Variational Inference for LDAlzhang/teach/6931a/slides/lda-zhou.pdf4 Variational Inference E-Step M-Step 5 Conclusion. Generative Process of LDAExponential FamilyNewton Method Variational

ADAVI: Automatic Dual Amortized Variational Inference ...

ADAVI: Automatic Dual Amortized Variational Inference ...

VARIATIONAL BAYESIAN PHYLOGENETIC INFERENCE

VARIATIONAL BAYESIAN PHYLOGENETIC INFERENCE

Collapsed Variational Bayesian Inference of the Author ...people.csail.mit.edu › ythomas › publications › 2016AuthorTopicCVB-… · Collapsed Variational Bayesian Inference

Collapsed Variational Bayesian Inference of the Author ...people.csail.mit.edu › ythomas › publications › 2016AuthorTopicCVB-… · Collapsed Variational Bayesian Inference

Variational Inference for Hierarchical Dirichlet Process ...cs.brown.edu/research/pubs/theses/ugrad/2015/stephenson.will.pdf · Variational Inference for Hierarchical Dirichlet Process

Variational Inference for Hierarchical Dirichlet Process ...cs.brown.edu/research/pubs/theses/ugrad/2015/stephenson.will.pdf · Variational Inference for Hierarchical Dirichlet Process