Graphical models, exponential families, and variational inference

DMCA

Graphical models, exponential families, and variational inference (2008)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Martin J. Wainwright , Michael I. Jordan

Citations:

784 - 26 self

BibTeX

@MISC{Wainwright08graphicalmodels,,
    author = {Martin J. Wainwright and Michael I. Jordan},
    title = {Graphical models, exponential families, and variational inference},
    year = {2008}
}

OpenURL

Abstract

The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances — including the key problems of computing marginals and modes of probability distributions — are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, we develop general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. We describe how a wide varietyof algorithms — among them sum-product, cluster variational methods, expectation-propagation, mean field methods, max-product and linear programming relaxation, as well as conic programming relaxations — can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.

Keyphrases

exponential family graphical model variational inference large-scale multivariate statistical model key problem linear programming relaxation unifying framework information retrieval variational approach communication theory many problem exponential family representation image processing mean field method markov chain monte carlo probability distribution complex dependency cumulant function approximate form general setting marginal probability probable configuration random variable variational representation specific instance statistical physic complementary alternative combinatorial optimization probabilistic graphical model general variational representation wide varietyof algorithm mathematical field approximation method variational method conjugate duality general source conic programming relaxation statistical machine large-scale statistical model

DMCA