Results 1  10
of
13
Counting in graph covers: a combinatorial characterization of the Bethe entropy function
 SUBMITTED TO IEEE TRANS. INF. THEORY
, 2012
"... ..."
(Show Context)
Bethe Bounds and Approximating the Global Optimum
"... Abstract—Inference in general Markov random fields (MRFs) is NPhard, though identifying the maximum a posteriori (MAP) configuration of pairwise MRFs with submodular cost functions is efficiently solvable using graph cuts. Marginal inference, however, even for this restricted class, is in #P. We pr ..."
Abstract

Cited by 9 (8 self)
 Add to MetaCart
(Show Context)
Abstract—Inference in general Markov random fields (MRFs) is NPhard, though identifying the maximum a posteriori (MAP) configuration of pairwise MRFs with submodular cost functions is efficiently solvable using graph cuts. Marginal inference, however, even for this restricted class, is in #P. We prove new formulations of derivatives of the Bethe free energy, provide bounds on the derivatives and bracket the locations of stationary points, introducing a new technique called Bethe bound propagation. Several results apply to pairwise models whether associative or not. Applying these to discretized pseudomarginals in the associative case we present a polynomial time approximation scheme for global optimization provided the maximum degree is O(log n), anddiscussseveralextensions. I.
On sampling from the gibbs distribution with random maximum aposteriori perturbations
 Advances in Neural Information Processing Systems
, 2013
"... In this paper we describe how MAP inference can be used to sample efficiently from Gibbs distributions. Specifically, we provide means for drawing either approximate or unbiased samples from Gibbs ’ distributions by introducing low dimensional perturbations and solving the corresponding MAP assign ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
(Show Context)
In this paper we describe how MAP inference can be used to sample efficiently from Gibbs distributions. Specifically, we provide means for drawing either approximate or unbiased samples from Gibbs ’ distributions by introducing low dimensional perturbations and solving the corresponding MAP assignments. Our approach also leads to new ways to derive lower bounds on partition functions. We demonstrate empirically that our method excels in the typical “high signalhigh coupling ” regime. The setting results in ragged energy landscapes that are challenging for alternative approaches to sampling and/or lower bounds. 1
Approximating marginals using discrete energy minimization
, 2012
"... classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specifi ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission.
Understanding the Bethe Approximation: When and How can it go Wrong?
"... Belief propagation is a remarkably effective tool for inference, even when applied to networks with cycles. It may be viewed as a way to seek the minimum of the Bethe free energy, though with no convergence guarantee in general. A variational perspective shows that, compared to exact inference, this ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
(Show Context)
Belief propagation is a remarkably effective tool for inference, even when applied to networks with cycles. It may be viewed as a way to seek the minimum of the Bethe free energy, though with no convergence guarantee in general. A variational perspective shows that, compared to exact inference, this minimization employs two forms of approximation: (i) the true entropy is approximated by the Bethe entropy, and (ii) the minimization is performed over a relaxation of the marginal polytope termed the local polytope. Here we explore when and how the Bethe approximation can fail for binary pairwise models by examining each aspect of the approximation, deriving results both analytically and with new experimental methods. 1
Approximating the Bethe partition function
"... When belief propagation (BP) converges, it does so to a stationary point of the Bethe free energy F, and is often strikingly accurate. However, it may converge only to a local optimum or may not converge at all. An algorithm was recently introduced by Weller and Jebara for attractive binary pairwise ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
When belief propagation (BP) converges, it does so to a stationary point of the Bethe free energy F, and is often strikingly accurate. However, it may converge only to a local optimum or may not converge at all. An algorithm was recently introduced by Weller and Jebara for attractive binary pairwise MRFs which is guaranteed to return an ɛapproximation to the global minimum of F in polynomial time provided the maximum degree ∆ = O(log n), where n is the number of variables. Here we extend their approach and derive a new method based on analyzing first derivatives of F, which leads to much better performance and, for attractive models, yields a fully polynomialtime approximation scheme (FPTAS) without any degree restriction. Further, our methods apply to general (nonattractive) models, though with no polynomial time guarantee in this case, demonstrating that approximating log of the Bethe partition function, log ZB = − min F, for a general model to additive ɛaccuracy may be reduced to a discrete MAP inference problem. This allows the merits of the global Bethe optimum to be tested.
The Bethe Free Energy Allows to Compute the Conditional Entropy of Graphical Code Instances. A Proof from the Polymer Expansion
, 2013
"... The main objective of this paper is to show that the Bethe free energy associated to a LowDensity ParityCheck code used over a Binary Symmetric Channel in a large noise regime is, with high probability, asymptotically exact as the block length grows. Using the loopsum as a starting point, we dev ..."
Abstract
 Add to MetaCart
The main objective of this paper is to show that the Bethe free energy associated to a LowDensity ParityCheck code used over a Binary Symmetric Channel in a large noise regime is, with high probability, asymptotically exact as the block length grows. Using the loopsum as a starting point, we develop new techniques based on the polymer expansion from statistical mechanics for general graphical models. The true free energy is computed as a series expansion containing the Bethe free energy (or entropy) as its zeroth order term plus a series of corrections. It is easily seen that convergence criteria for such expansions are satisfied for general hightemperature models. In particular, when the graphical model has large girth the Bethe free energy is asymptotically exact. We apply these general results to ensembles of LowDensity GeneratorMatrix and ParityCheck codes. While the application to GeneratorMatrix codes is quite straightforward, the case of ParityCheck codes requires nontrivial extra ideas because the hard constraints correspond to a low temperature regime. Nevertheless one can combine the polymer expansion with expander and counting arguments to show that the difference between the true and Bethe free energies vanishes with high probability in the large block length limit.
Bethe and Related Pairwise Entropy Approximations
"... For undirected graphical models, belief propagation often performs remarkably well for approximate marginal inference, and may be viewed as a heuristic to minimize the Bethe free energy. Focusing on binary pairwise models, we demonstrate that several recent results on the Bethe approximation ma ..."
Abstract
 Add to MetaCart
(Show Context)
For undirected graphical models, belief propagation often performs remarkably well for approximate marginal inference, and may be viewed as a heuristic to minimize the Bethe free energy. Focusing on binary pairwise models, we demonstrate that several recent results on the Bethe approximation may be generalized to a broad family of related pairwise free energy approximations with arbitrary counting numbers. We explore approximation error and shed light on the empirical success of the Bethe approximation. 1
Motivation: undirected graphical models Example: Part of epinions social network (mixed)
, 2015
"... Motivation: undirected graphical models Powerful way to represent relationships across variables Many applications including: computer vision, social network analysis, deep belief networks, protein folding... In this talk, focus on binary pairwise (Ising) models ..."
Abstract
 Add to MetaCart
(Show Context)
Motivation: undirected graphical models Powerful way to represent relationships across variables Many applications including: computer vision, social network analysis, deep belief networks, protein folding... In this talk, focus on binary pairwise (Ising) models
Approximating Marginals Using Discrete Energy Minimization
"... We consider the problem of inference in a graphical model with binary variables. While in theory it is arguably preferable to compute marginal probabilities, in practice researchers often use MAP inference due to the availability of efficient discrete optimization algorithms. We bridge the gap be ..."
Abstract
 Add to MetaCart
We consider the problem of inference in a graphical model with binary variables. While in theory it is arguably preferable to compute marginal probabilities, in practice researchers often use MAP inference due to the availability of efficient discrete optimization algorithms. We bridge the gap between the two approaches by introducing the Discrete Marginals technique in which approximate marginals are obtained by minimizing an objective function with unary and pairwise terms over a discretized domain. This allows the use of techniques originally developed for MAPMRF inference and learning. We explore two ways to set up the objective function by discretizing the Bethe free energy and by learning it from training data. Experimental results show that for certain types of graphs a learned function can outperform the Bethe approximation. We also establish a link between the Bethe free energy and submodular functions. 1.