Variational Bayes Model Selection for Mixture Distribution

Click here to load reader

  • date post

    14-Feb-2016
  • Category

    Documents

  • view

    37
  • download

    1

Embed Size (px)

description

Variational Bayes Model Selection for Mixture Distribution. Authors: Adrian Corduneanu & Christopher M. Bishop. Presented by Shihao Ji Duke University Machine Learning Group Jan. 20, 2006 . Outline. Introduction – model selection Automatic Relevance Determination (ARD) Experimental Results - PowerPoint PPT Presentation

Transcript of Variational Bayes Model Selection for Mixture Distribution

  • Variational Bayes Model Selectionfor Mixture DistributionPresented by Shihao JiDuke University Machine Learning GroupJan. 20, 2006 Authors: Adrian Corduneanu & Christopher M. Bishop

  • Introduction model selection

    Automatic Relevance Determination (ARD)

    Experimental Results

    Application to HMMs

    Outline

  • Cross validation

    Bayesian approaches

    MCMC and Laplace approximation

    (Traditional) variational method

    (Type II) variational method

    Introduction

  • relevance vector regression

    Given a dataset , we assume is Gaussian

    Automatic Relevance Determination (ARD)Determination of hyperparameters:Likelihood:Prior:Posterior:Type II ML

  • mixture of Gaussian

    Given an observed dataset , we assume each data point is drawn independently from a mixture of Gaussian density

    Likelihood:Prior:Posterior:Determination of mixing coefficients:VBType II MLAutomatic Relevance Determination (ARD)

  • Bayesian method: , model selection

    Automatic Relevance Determination (ARD)Component elimination: if ,

    i.e.,

  • Experimental Results600 points drawn from a mixture of 5 Gaussians.Bayesian method vs. cross-validation

  • Initially the model had 15 mixtures, finally was pruned down to 3 mixturesComponent elimination

    Experimental Results

  • Experimental Results

  • hidden Markov model

    Given an observed dataset , we assume each data sequence is generated independently from an HMM

    Likelihood:Prior:Posterior:Determination of p and A:VBType II MLAutomatic Relevance Determination (ARD)

  • Define -- visiting frequency

    whereBayesian method: , model selection

    Automatic Relevance Determination (ARD)State elimination: if ,

  • Experimental Results (1)

  • Experimental Results (2)

  • Experimental Results (3)

  • Questions?