Lecture : Variational Inference

Riashat IslamSlides courtesy of David Blei

Reasoning and Learning LabMcGill University

31st October 2018

Probabilistic Machine Learning

Approximate Inference

Computational Challenge

Probabilistic Model

Variational Inference

Variational Bound

Variational Bound

Motivation : Topic Modelling

Example : Latent Dirichlet Allocation

Example : Latent Dirichlet Allocation

Example : Latent Dirichlet Allocation

LDA as Graphical Model

Posterior Inference in LDA

The Evidence Lower Bound (ELBO)

Stochastic Gradients of ELBO

Variational Inference Recipe

Variational Inference Recipe

Variational Inference Recipe

Variational Inference Recipe

Variational Inference Recipe

Variational Inference Recipe

Variational Inference Recipe

Black Box Variational Inference

Problem of Classical VI

New VI Recipe

Computing Gradients of Expectations

Score Function Gradients of ELBO

Score Function Estimator

Noisy Unbiased Gradients

Black Box Variational Inference

Pathwise Gradients of ELBO

Pathwise Estimator

Pathwise Estimator

Score Function vs Pathwise Estimator