DMCA
Variational algorithms for approximate Bayesian inference (2003)
| Citations: | 440 - 9 self |
BibTeX
@TECHREPORT{Beal03variationalalgorithms,
author = {Matthew J. Beal},
title = {Variational algorithms for approximate Bayesian inference},
institution = {},
year = {2003}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
The Bayesian framework for machine learning allows for the incorporation of prior knowledge in a coherent way, avoids overfitting problems, and provides a principled basis for selecting between alternative models. Unfortunately the computations required are usually intractable. This thesis presents a unified variational Bayesian (VB) framework which approximates these computations in models with latent variables using a lower bound on the marginal likelihood. Chapter 1 presents background material on Bayesian inference, graphical models, and propaga-tion algorithms. Chapter 2 forms the theoretical core of the thesis, generalising the expectation-maximisation (EM) algorithm for learning maximum likelihood parameters to the VB EM al-gorithm which integrates over model parameters. The algorithm is then specialised to the large family of conjugate-exponential (CE) graphical models, and several theorems are presented to pave the road for automated VB derivation procedures in both directed and undirected graphs (Bayesian and Markov networks, respectively). Chapters 3-5 derive and apply the VB EM algorithm to three commonly-used and important models: mixtures of factor analysers, linear dynamical systems, and hidden Markov models.
Keyphrases
variational algorithm approximate bayesian inference graphical model machine learning maximum likelihood parameter important model automated vb derivation procedure coherent way vb em algorithm propaga-tion algorithm principled basis alternative model chapter 3-5 derive linear dynamical system theoretical core undirected graph several theorem bayesian inference model parameter vb em al-gorithm latent variable factor analyser hidden markov model large family prior knowledge bayesian framework marginal likelihood markov network unified variational bayesian
