Results 1  10
of
96
Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms
 IEEE Transactions on Information Theory
, 2005
"... Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems t ..."
Abstract

Cited by 585 (13 self)
 Add to MetaCart
(Show Context)
Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain regionbased free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a “valid ” or “maxentnormal ” approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the “Bethe method, ” the “junction graph method, ” the “cluster variation method, ” and the “region graph method.” Finally, we explain how to tell whether a regionbased approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.
Towards understanding and harnessing the potential of clause learning
 Journal of Artificial Intelligence Research
, 2004
"... Efficient implementations of DPLL with the addition of clause learning are the fastest complete Boolean satisfiability solvers and can handle many significant realworld problems, such as verification, planning and design. Despite its importance, little is known of the ultimate strengths and limitat ..."
Abstract

Cited by 99 (10 self)
 Add to MetaCart
(Show Context)
Efficient implementations of DPLL with the addition of clause learning are the fastest complete Boolean satisfiability solvers and can handle many significant realworld problems, such as verification, planning and design. Despite its importance, little is known of the ultimate strengths and limitations of the technique. This paper presents the first precise characterization of clause learning as a proof system (CL), and begins the task of understanding its power by relating it to the wellstudied resolution proof system. In particular, we show that with a new learning scheme, CL can provide exponentially shorter proofs than many proper refinements of general resolution (RES) satisfying a natural property. These include regular and DavisPutnam resolution, which are already known to be much stronger than ordinary DPLL. We also show that a slight variant of CL with unlimited restarts is as powerful as RES itself. Translating these analytical results to practice, however, presents a challenge because of the nondeterministic nature of clause learning algorithms. We propose a novel way of exploiting the underlying problem structure, in the form of a high level problem description such as a graph or PDDL specification, to guide clause learning algorithms toward faster solutions. We show that this leads to exponential speedups on grid and randomized pebbling problems, as well as substantial improvements on certain ordering formulas. 1.
The Probabilistic Analysis of a Greedy Satisfiability Algorithm
, 2002
"... Consider the following simple, greedy DavisPutnam algorithm applied to a random 3CNF formula of fixed density (clauses to variables ratio): Arbitrarily select and set to True a literal that appears in as many clauses as possible, irrespective of their size (and irrespective of the number of occu ..."
Abstract

Cited by 76 (6 self)
 Add to MetaCart
(Show Context)
Consider the following simple, greedy DavisPutnam algorithm applied to a random 3CNF formula of fixed density (clauses to variables ratio): Arbitrarily select and set to True a literal that appears in as many clauses as possible, irrespective of their size (and irrespective of the number of occurrences of the negation of the literal). Delete these clauses from the formula, and also delete the negation of this literal from any clauses it appears. Repeat. If however unit clauses ever appear, then first repeatedly and in any order set the literals in them to True and delete and shrink clauses accordingly, until no unit clause remains. Also if at any step an empty clause appears, then do not backtrack, but just terminate the algorithm and report failure. A slight modification of this algorithm is probabilistically analyzed in this paper (rigorously). It is proved that for random formulas of n variables and density up to 3.42, it succeeds in producing a satisfying truth assignment with bounded away from zero probability, as n approaches infinity. Therefore the satisfiability threshold is at least 3.42.
On the solutionspace geometry of random constraint satisfaction problems
 In STOC ’06: Proceedings of the thirtyeighth annual ACM symposium on Theory of computing
, 2006
"... For a large number of random constraint satisfaction problems, such as random kSAT and random graph and hypergraph coloring, there are very good estimates of the largest constraint density for which solutions exist. Yet, all known polynomialtime algorithms for these problems fail to find solutions ..."
Abstract

Cited by 61 (2 self)
 Add to MetaCart
(Show Context)
For a large number of random constraint satisfaction problems, such as random kSAT and random graph and hypergraph coloring, there are very good estimates of the largest constraint density for which solutions exist. Yet, all known polynomialtime algorithms for these problems fail to find solutions even at much lower densities. To understand the origin of this gap we study how the structure of the space of solutions evolves in such problems as constraints are added. In particular, we prove that much before solutions disappear, they organize into an exponential number of clusters, each of which is relatively small and far apart from all other clusters. Moreover, inside each cluster most variables are frozen, i.e., take only one value. The existence of such frozen variables gives a satisfying intuitive explanation for the failure of the polynomialtime algorithms analyzed so far. At the same time, our results establish rigorously one of the two main hypotheses underlying Survey Propagation, a heuristic introduced by physicists in recent years that appears to perform extraordinarily well on random constraint satisfaction problems. 1
Satisfiability Solvers
, 2008
"... The past few years have seen an enormous progress in the performance of Boolean satisfiability (SAT) solvers. Despite the worstcase exponential run time of all known algorithms, satisfiability solvers are increasingly leaving their mark as a generalpurpose tool in areas as diverse as software and h ..."
Abstract

Cited by 50 (0 self)
 Add to MetaCart
The past few years have seen an enormous progress in the performance of Boolean satisfiability (SAT) solvers. Despite the worstcase exponential run time of all known algorithms, satisfiability solvers are increasingly leaving their mark as a generalpurpose tool in areas as diverse as software and hardware verification [29–31, 228], automatic test pattern generation [138, 221], planning [129, 197], scheduling [103], and even challenging problems from algebra [238]. Annual SAT competitions have led to the development of dozens of clever implementations of such solvers [e.g. 13,
Constraint satisfaction problems and neural networks: statistical physics approach
 J. Physiol. Paris
, 2009
"... A new field of research is rapidly expanding at the crossroad between statistical physics, information theory and combinatorial optimization. In particular, the use of cutting edge statistical physics concepts and methods allow one to solve very large constraint satisfaction problems like random sat ..."
Abstract

Cited by 27 (0 self)
 Add to MetaCart
(Show Context)
A new field of research is rapidly expanding at the crossroad between statistical physics, information theory and combinatorial optimization. In particular, the use of cutting edge statistical physics concepts and methods allow one to solve very large constraint satisfaction problems like random satisfiability, coloring, or error correction. Several aspects of these developments should be relevant for the understanding of functional complexity in neural networks. On the one hand the message passing procedures which are used in these new algorithms are based on local exchange of information, and succeed in solving some of the hardest computational problems. On the other hand some crucial inference problems in neurobiology, like those generated in multielectrode recordings, naturally translate into hard constraint satisfaction problems. This paper gives a nontechnical introduction to this field, emphasizing the main ideas at work in message passing strategies and their possible relevance to neural networks modelling. It also introduces a new message passing algorithm for inferring interactions between variables from correlation data, which could be useful in the analysis of multielectrode recording data.
Solving constraint satisfaction problems through belief propagationguided decimation
 in Proc. of the Allerton Conf. on Commun., Control, and Computing
"... Abstract — Message passing algorithms have proved surprisingly successful in solving hard constraint satisfaction problems on sparse random graphs. In such applications, variables are fixed sequentially to satisfy the constraints. Message passing is run after each step. Its outcome provides an heuri ..."
Abstract

Cited by 20 (2 self)
 Add to MetaCart
Abstract — Message passing algorithms have proved surprisingly successful in solving hard constraint satisfaction problems on sparse random graphs. In such applications, variables are fixed sequentially to satisfy the constraints. Message passing is run after each step. Its outcome provides an heuristic to make choices at next step. This approach has been referred to as ‘decimation, ’ with reference to analogous procedures in statistical physics. The behavior of decimation procedures is poorly understood. Here we consider a simple randomized decimation algorithm based on belief propagation (BP), and analyze its behavior on random ksatisfiability formulae. In particular, we propose a tree model for its analysis and we conjecture that it provides asymptotically exact predictions in the limit of large instances. This conjecture is confirmed by numerical simulations. I.
The number of matchings in random graphs
 Journal of Statistical Mechanics: Theory and Experiment
"... Abstract. We study matchings on sparse random graphs by means of the cavity method. ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
(Show Context)
Abstract. We study matchings on sparse random graphs by means of the cavity method.
Using Problem Structure for Efficient Clause Learning
 In Proceedings of the 6th International Conference on Theory and Applications of Satisfiability Testing
, 2003
"... DPLL based clause learning algorithms for satisfiability testing are known to work very well in practice. However, like most branchandbound techniques, their performance depends heavily on the variable order used in making branching decisions. We propose a novel way of exploiting the underlying ..."
Abstract

Cited by 14 (4 self)
 Add to MetaCart
(Show Context)
DPLL based clause learning algorithms for satisfiability testing are known to work very well in practice. However, like most branchandbound techniques, their performance depends heavily on the variable order used in making branching decisions. We propose a novel way of exploiting the underlying problem structure to guide clause learning algorithms toward faster solutions. The key idea is to use a higher level problem description, such as a graph or a PDDL specification, to generate a good branching sequence as an aid to SAT solvers.
A rigorous proof of the cavity method for counting matchings
"... In this paper we rigorously prove the validity of the cavity method for the problem of counting the number of matchings in graphs with large girth. Cavity method is an important heuristic developed by statistical physicists that has lead to the development of faster distributed algorithms for probl ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
In this paper we rigorously prove the validity of the cavity method for the problem of counting the number of matchings in graphs with large girth. Cavity method is an important heuristic developed by statistical physicists that has lead to the development of faster distributed algorithms for problems in various combinatorial optimization problems. The validity of the approach has been supported mostly by numerical simulations. In this paper we prove the validity of cavity method for the problem of counting matchings using rigorous techniques. We hope that these rigorous approaches will finally help us establish the validity of the cavity method in general.