Results 1 
5 of
5
Bethe Bounds and Approximating the Global Optimum
"... Abstract—Inference in general Markov random fields (MRFs) is NPhard, though identifying the maximum a posteriori (MAP) configuration of pairwise MRFs with submodular cost functions is efficiently solvable using graph cuts. Marginal inference, however, even for this restricted class, is in #P. We pr ..."
Abstract

Cited by 9 (8 self)
 Add to MetaCart
(Show Context)
Abstract—Inference in general Markov random fields (MRFs) is NPhard, though identifying the maximum a posteriori (MAP) configuration of pairwise MRFs with submodular cost functions is efficiently solvable using graph cuts. Marginal inference, however, even for this restricted class, is in #P. We prove new formulations of derivatives of the Bethe free energy, provide bounds on the derivatives and bracket the locations of stationary points, introducing a new technique called Bethe bound propagation. Several results apply to pairwise models whether associative or not. Applying these to discretized pseudomarginals in the associative case we present a polynomial time approximation scheme for global optimization provided the maximum degree is O(log n), anddiscussseveralextensions. I.
Understanding the Bethe Approximation: When and How can it go Wrong?
"... Belief propagation is a remarkably effective tool for inference, even when applied to networks with cycles. It may be viewed as a way to seek the minimum of the Bethe free energy, though with no convergence guarantee in general. A variational perspective shows that, compared to exact inference, this ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
(Show Context)
Belief propagation is a remarkably effective tool for inference, even when applied to networks with cycles. It may be viewed as a way to seek the minimum of the Bethe free energy, though with no convergence guarantee in general. A variational perspective shows that, compared to exact inference, this minimization employs two forms of approximation: (i) the true entropy is approximated by the Bethe entropy, and (ii) the minimization is performed over a relaxation of the marginal polytope termed the local polytope. Here we explore when and how the Bethe approximation can fail for binary pairwise models by examining each aspect of the approximation, deriving results both analytically and with new experimental methods. 1
Approximating the Bethe partition function
"... When belief propagation (BP) converges, it does so to a stationary point of the Bethe free energy F, and is often strikingly accurate. However, it may converge only to a local optimum or may not converge at all. An algorithm was recently introduced by Weller and Jebara for attractive binary pairwise ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
When belief propagation (BP) converges, it does so to a stationary point of the Bethe free energy F, and is often strikingly accurate. However, it may converge only to a local optimum or may not converge at all. An algorithm was recently introduced by Weller and Jebara for attractive binary pairwise MRFs which is guaranteed to return an ɛapproximation to the global minimum of F in polynomial time provided the maximum degree ∆ = O(log n), where n is the number of variables. Here we extend their approach and derive a new method based on analyzing first derivatives of F, which leads to much better performance and, for attractive models, yields a fully polynomialtime approximation scheme (FPTAS) without any degree restriction. Further, our methods apply to general (nonattractive) models, though with no polynomial time guarantee in this case, demonstrating that approximating log of the Bethe partition function, log ZB = − min F, for a general model to additive ɛaccuracy may be reduced to a discrete MAP inference problem. This allows the merits of the global Bethe optimum to be tested.
HIGHER ORDER MARKOV RANDOM FIELDS FOR INDEPENDENT SETS
"... It is wellknown that if one samples from the independent sets of a large regular graph of large girth using a pairwise Markov random eld (i.e. hardcore model) in the uniqueness regime, each excluded node has a binomially distributed number of included neighbors in the limit. In this paper, motivate ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
It is wellknown that if one samples from the independent sets of a large regular graph of large girth using a pairwise Markov random eld (i.e. hardcore model) in the uniqueness regime, each excluded node has a binomially distributed number of included neighbors in the limit. In this paper, motivated by applications to the design of communication networks, we pose the question of how to sample from the independent sets of such a graph so that the number of included neighbors of each excluded node has a dierent distribution of our choosing. We observe that higher order Markov random elds are wellsuited to this task, and investigate the properties of these models. For the family of socalled reverse ultra logconcave distributions, which includes the truncated Poisson and geometric, we give necessary and sucient conditions for the natural higher order Markov random eld which induces the desired distribution to be in the uniqueness regime, in terms of the set of solutions to a certain system of equations. We also show that these Markov random elds undergo a phase transition, and give explicit bounds on the associated critical activity, which we prove to exhibit a certain robustness. For distributions which are small perturbations around the binomial distribution realized by the hardcore model at critical activity, we give a description of the corresponding uniqueness regime in terms of a simple polyhedral cone. Our analysis reveals an interesting nonmonotonicity with regards to biasing towards excluded nodes with no included neighbors. We conclude with a broader discussion of the potential use of higher order Markov random elds for analyzing independent sets in graphs. 1. Introduction. Recently
Bethe and Related Pairwise Entropy Approximations
"... For undirected graphical models, belief propagation often performs remarkably well for approximate marginal inference, and may be viewed as a heuristic to minimize the Bethe free energy. Focusing on binary pairwise models, we demonstrate that several recent results on the Bethe approximation ma ..."
Abstract
 Add to MetaCart
(Show Context)
For undirected graphical models, belief propagation often performs remarkably well for approximate marginal inference, and may be viewed as a heuristic to minimize the Bethe free energy. Focusing on binary pairwise models, we demonstrate that several recent results on the Bethe approximation may be generalized to a broad family of related pairwise free energy approximations with arbitrary counting numbers. We explore approximation error and shed light on the empirical success of the Bethe approximation. 1