DMCA
Expectation consistent approximate inference (2005)
| Venue: | JOURNAL OF MACHINE LEARNING RESEARCH |
| Citations: | 33 - 5 self |
BibTeX
@ARTICLE{Opper05expectationconsistent,
author = {Manfred Opper and Ole Winther},
title = {Expectation consistent approximate inference},
journal = {JOURNAL OF MACHINE LEARNING RESEARCH},
year = {2005},
number = {2117},
pages = {2177--2204}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We propose a novel framework for approximations to intractable probabilistic models which is based on a free energy formulation. The approximation can be understood from replacing an average over the original intractable distribution with a tractable one. It requires two tractable probability distributions which are made consistent on a set of moments and encode different features of the original intractable distribution. In this way we are able to use Gaussian approximations for models with discrete or bounded variables which allow us to include non-trivial correlations which are neglected in many other methods. We test the framework on toy benchmark problems for binary variables on fully connected graphs and 2D grids and compare with other methods, such as loopy belief propagation. Good performance is already achieved by using single nodes as tractable substructures. Significant improvements are obtained when a spanning tree is used instead.
Keyphrases
expectation consistent approximate inference original intractable distribution good performance significant improvement single node toy benchmark problem intractable probabilistic model novel framework non-trivial correlation tractable substructure binary variable free energy formulation loopy belief propagation spanning tree many method different feature tractable probability distribution gaussian approximation tractable one
