Results 1 
3 of
3
Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms
 IEEE Transactions on Information Theory
, 2005
"... Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems t ..."
Abstract

Cited by 585 (13 self)
 Add to MetaCart
(Show Context)
Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain regionbased free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a “valid ” or “maxentnormal ” approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the “Bethe method, ” the “junction graph method, ” the “cluster variation method, ” and the “region graph method.” Finally, we explain how to tell whether a regionbased approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.
Bethe free energy, Kikuchi approximations and belief propagation algorithms
, 2000
"... Belief propagation (BP) was only supposed to work for treelike networks but works surprisingly well in many applications involving networks with loops, including turbo codes. However, there has been little understanding of the algorithm or the nature of the solutions it nds for general graphs. ..."
Abstract

Cited by 95 (2 self)
 Add to MetaCart
Belief propagation (BP) was only supposed to work for treelike networks but works surprisingly well in many applications involving networks with loops, including turbo codes. However, there has been little understanding of the algorithm or the nature of the solutions it nds for general graphs. We show that BP can only converge to a stationary point of an approximate free energy, known as the Bethe free energy in statistical physics. This result characterizes BP xedpoints and makes connections with variational approaches to approximate inference. More importantly, our analysis lets us build on the progress made in statistical physics since Bethe's approximation was introduced in 1935. Kikuchi and others have shown how to construct more accurate free energy approximations, of which Bethe's approximation is the simplest. Exploiting the insights from our analysis, we derive generalized belief propagation (GBP) versions of these Kikuchi approximations. These new message passing algorithms can be signicantly more accurate than ordinary BP, at an adjustable increase in complexity. We illustrate such a new GBP algorithm on a grid Markov network and show that it gives much more accurate marginal probabilities than those found using ordinary BP.
Selected problems in lattice statistical mechanics
, 2005
"... This thesis consists of an introduction and four chapters, with each chapter covering a different topic. In the introduction, we introduce the models that we study in the later chapters. In Chapter 2, we study corner transfer matrices as a method for generating series expansions in statistical mecha ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
This thesis consists of an introduction and four chapters, with each chapter covering a different topic. In the introduction, we introduce the models that we study in the later chapters. In Chapter 2, we study corner transfer matrices as a method for generating series expansions in statistical mechanical models. This is based on the work of Baxter, whose CTM equations we rederive in detail. We then propose two methods that utilise these CTM equations to derive series expansions. The first one is based on iterating through the equations sequentially. We ran this algorithm on the hard squares model and produced 48 series terms. The second method, based on the corner transfer matrix renormalization group method of Nishino and Okunishi, is much faster (though still exponential in time), but only currently works for numerical calculations. In Chapter 3, we apply the nite lattice method and our renormalization group corner transfer matrix method to the Ising model with secondnearestneighbour interactions. In particular, we study the crossover exponent of the critical line near the point J1 = 0; tanh J2 kT