Results 1  10
of
12
Complexity of Bethe approximation
 In Artificial Intelligence and Statistics
, 2012
"... This paper resolves a common complexity issue in the Bethe approximation of statistical physics and the sumproduct Belief Propagation (BP) algorithm of artificial intelligence. The Bethe approximation reduces the problem of computing the partition function in a graphical model to that of solving a ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
This paper resolves a common complexity issue in the Bethe approximation of statistical physics and the sumproduct Belief Propagation (BP) algorithm of artificial intelligence. The Bethe approximation reduces the problem of computing the partition function in a graphical model to that of solving a set of nonlinear equations, socalled the Bethe equation. On the other hand, the BP algorithm is a popular heuristic method for estimating marginal distribution in a graphical model. Although they are inspired and developed from different directions, Yedidia, Freeman and Weiss (2004) established a somewhat surprising connection: the BP algorithm solves the Bethe equation if it converges (however, it often does not). This naturally motivates the following important question to understand their limitations and empirical successes: the Bethe equation is computationally easy to solve? We present a message passing algorithm solving the Bethe equation in polynomial number of bitwise operations for arbitrary binary graphical models of n nodes where the maximum degree in the underlying graph is O(log n). Our algorithm, an alternative to BP fixing its convergence issue, is the first fully polynomialtime approximation scheme for the BP fixed point computation in such a large class of graphical models. Moreover, we believe that our technique is of broader interest to understand the computational complexity of the cavity method in statistical physics.
Approximating the Bethe partition function
"... When belief propagation (BP) converges, it does so to a stationary point of the Bethe free energy F, and is often strikingly accurate. However, it may converge only to a local optimum or may not converge at all. An algorithm was recently introduced by Weller and Jebara for attractive binary pairwise ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
When belief propagation (BP) converges, it does so to a stationary point of the Bethe free energy F, and is often strikingly accurate. However, it may converge only to a local optimum or may not converge at all. An algorithm was recently introduced by Weller and Jebara for attractive binary pairwise MRFs which is guaranteed to return an ɛapproximation to the global minimum of F in polynomial time provided the maximum degree ∆ = O(log n), where n is the number of variables. Here we extend their approach and derive a new method based on analyzing first derivatives of F, which leads to much better performance and, for attractive models, yields a fully polynomialtime approximation scheme (FPTAS) without any degree restriction. Further, our methods apply to general (nonattractive) models, though with no polynomial time guarantee in this case, demonstrating that approximating log of the Bethe partition function, log ZB = − min F, for a general model to additive ɛaccuracy may be reduced to a discrete MAP inference problem. This allows the merits of the global Bethe optimum to be tested.
Approximately Counting the Number of Constrained Arrays via the SumProduct Algorithm
"... Abstract—Very often, constrained coding schemes impose nonlinear constraints on arrays and therefore it is usually nontrivial to determine how many arrays satisfy the given constraints. In this paper we show that there are nontrivial constrained coding scenarios where the number of constrained arra ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract—Very often, constrained coding schemes impose nonlinear constraints on arrays and therefore it is usually nontrivial to determine how many arrays satisfy the given constraints. In this paper we show that there are nontrivial constrained coding scenarios where the number of constrained arrays can be estimated to a surprisingly high accuracy with the help of the sumproduct algorithm, despite the fact that the underlying factor graphs have many short cycles, and we investigate the reasons why this is the case. These findings open up interesting possibilities for determining the number of constrained arrays also for scenarios where other counting and bounding techniques are not readily available. I.
Independent sets from an algebraic perspective
 INTERNAT. J. ALGEBRA COMPUT
, 2011
"... In this paper, we study the basic problem of counting independent sets in a graph and, in particular, the problem of counting antichains in a finite poset, from an algebraic perspective. We show that neither independence polynomials of bipartite CohenMacaulay graphs nor Hilbert series of initial i ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
In this paper, we study the basic problem of counting independent sets in a graph and, in particular, the problem of counting antichains in a finite poset, from an algebraic perspective. We show that neither independence polynomials of bipartite CohenMacaulay graphs nor Hilbert series of initial ideals of radical zerodimensional complete intersections ideals, can be evaluated in polynomial time, unless #P = P. Moreover, we present a family of radical zerodimensional complete intersection ideals JP associated to a finite poset P, for which we describe a universal Gröbner basis. This implies that the bottleneck in computing the dimension of the quotient by JP (that is, the number of zeros of JP) using Gröbner methods lies in the description of the standard monomials.
Loop Calculus for NonBinary Alphabets using Concepts from Information Geometry
, 2015
"... The Bethe approximation is a wellknown approximation of the partition function used in statistical physics. Recently, an equality relating the partition function and its Bethe approximation was obtained for graphical models with binary variables by Chertkov and Chernyak. In this equality, the mult ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
The Bethe approximation is a wellknown approximation of the partition function used in statistical physics. Recently, an equality relating the partition function and its Bethe approximation was obtained for graphical models with binary variables by Chertkov and Chernyak. In this equality, the multiplicative error in the Bethe approximation is represented as a weighted sum over all generalized loops in the graphical model. In this paper, the equality is generalized to graphical models with nonbinary alphabet using concepts from information geometry.
HIGHER ORDER MARKOV RANDOM FIELDS FOR INDEPENDENT SETS
"... It is wellknown that if one samples from the independent sets of a large regular graph of large girth using a pairwise Markov random eld (i.e. hardcore model) in the uniqueness regime, each excluded node has a binomially distributed number of included neighbors in the limit. In this paper, motivate ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
It is wellknown that if one samples from the independent sets of a large regular graph of large girth using a pairwise Markov random eld (i.e. hardcore model) in the uniqueness regime, each excluded node has a binomially distributed number of included neighbors in the limit. In this paper, motivated by applications to the design of communication networks, we pose the question of how to sample from the independent sets of such a graph so that the number of included neighbors of each excluded node has a dierent distribution of our choosing. We observe that higher order Markov random elds are wellsuited to this task, and investigate the properties of these models. For the family of socalled reverse ultra logconcave distributions, which includes the truncated Poisson and geometric, we give necessary and sucient conditions for the natural higher order Markov random eld which induces the desired distribution to be in the uniqueness regime, in terms of the set of solutions to a certain system of equations. We also show that these Markov random elds undergo a phase transition, and give explicit bounds on the associated critical activity, which we prove to exhibit a certain robustness. For distributions which are small perturbations around the binomial distribution realized by the hardcore model at critical activity, we give a description of the corresponding uniqueness regime in terms of a simple polyhedral cone. Our analysis reveals an interesting nonmonotonicity with regards to biasing towards excluded nodes with no included neighbors. We conclude with a broader discussion of the potential use of higher order Markov random elds for analyzing independent sets in graphs. 1. Introduction. Recently
The Bethe Free Energy Allows to Compute the Conditional Entropy of Graphical Code Instances. A Proof from the Polymer Expansion
, 2013
"... The main objective of this paper is to show that the Bethe free energy associated to a LowDensity ParityCheck code used over a Binary Symmetric Channel in a large noise regime is, with high probability, asymptotically exact as the block length grows. Using the loopsum as a starting point, we dev ..."
Abstract
 Add to MetaCart
The main objective of this paper is to show that the Bethe free energy associated to a LowDensity ParityCheck code used over a Binary Symmetric Channel in a large noise regime is, with high probability, asymptotically exact as the block length grows. Using the loopsum as a starting point, we develop new techniques based on the polymer expansion from statistical mechanics for general graphical models. The true free energy is computed as a series expansion containing the Bethe free energy (or entropy) as its zeroth order term plus a series of corrections. It is easily seen that convergence criteria for such expansions are satisfied for general hightemperature models. In particular, when the graphical model has large girth the Bethe free energy is asymptotically exact. We apply these general results to ensembles of LowDensity GeneratorMatrix and ParityCheck codes. While the application to GeneratorMatrix codes is quite straightforward, the case of ParityCheck codes requires nontrivial extra ideas because the hard constraints correspond to a low temperature regime. Nevertheless one can combine the polymer expansion with expander and counting arguments to show that the difference between the true and Bethe free energies vanishes with high probability in the large block length limit.
Research Statement
"... the relationships among a large number of interacting entities. In a model selection setting we may wish to learn a “simple ” statistical model to approximate the behavior observed in a collection of random variables. Modern data analysis tasks in geophysics, economics, and image processing often in ..."
Abstract
 Add to MetaCart
(Show Context)
the relationships among a large number of interacting entities. In a model selection setting we may wish to learn a “simple ” statistical model to approximate the behavior observed in a collection of random variables. Modern data analysis tasks in geophysics, economics, and image processing often involve learning statistical models over collections of random variables that may number in the hundreds of thousands, or even a few million. In a computational biology setting a typical question involving gene regulatory networks is to discover the interaction patterns among a collection of genes in order to better understand how a gene influences or is influenced by other genes. Similar problems also arise in the analysis of biological, social, or chemical reaction networks in which one seeks to better understand a complicated network by decomposing it into simpler networks. Models based on graphs offer a fruitful framework to solve such problems, as graphs often provide a concise representation of the interactions among a large set of variables. Motivated by these considerations my overall research agenda is to develop new statistical and computational foundations to reason in a unified way about graphs and statistics. This approach has already led to original and exciting convex optimization methods for solving challenging problems in signal processing and machine learning. Graphical models. The first direction is the study of graphical models, in which a statistical model is