DMCA
Variational bayesian inference with stochastic search (2012)
Cached
Download Links
| Venue: | In ICML |
| Citations: | 25 - 4 self |
BibTeX
@INPROCEEDINGS{Paisley12variationalbayesian,
author = {John Paisley and David M. Blei and Michael I. Jordan},
title = {Variational bayesian inference with stochastic search},
booktitle = {In ICML},
year = {2012}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by max-imizing a lower bound on the marginal likeli-hood. This requires the ability to integrate a sum of terms in the log joint likelihood using this factorized distribution. Often not all in-tegrals are in closed form, which is typically handled by using a lower bound. We present an alternative algorithm based on stochastic optimization that allows for direct optimiza-tion of the variational lower bound. This method uses control variates to reduce the variance of the stochastic search gradient, in which existing lower bounds can play an im-portant role. We demonstrate the approach on two non-conjugate models: logistic regres-sion and an approximation to the HDP. 1.
Keyphrases
stochastic search variational bayesian inference non-conjugate model stochastic search gradient mean-field variational inference approximate bayesian posterior inference marginal likeli-hood stochastic optimization factorized distribution factorized set logistic regres-sion log joint likelihood closed form alternative algorithm im-portant role full posterior distribution direct optimiza-tion
