Results 1  10
of
81
Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms
 IEEE Transactions on Information Theory
, 2005
"... Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems t ..."
Abstract

Cited by 585 (13 self)
 Add to MetaCart
(Show Context)
Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain regionbased free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a “valid ” or “maxentnormal ” approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the “Bethe method, ” the “junction graph method, ” the “cluster variation method, ” and the “region graph method.” Finally, we explain how to tell whether a regionbased approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.
ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context,”
 BMC Bioinformatics,
, 2006
"... ..."
(Show Context)
Free energy estimates of allatom protein structures using generalized belief propagation
 IN PROCEEDINGS OF THE 11TH ANNUAL INTERNATIONAL CONFERENCE ON RESEARCH IN COMPUTATIONAL MOLECULAR BIOLOGY
, 2007
"... We present a technique for approximating the free energy of protein structures using Generalized Belief Propagation (GBP). The accuracy and utility of these estimates are then demonstrated in two different application domains. First, we show that the entropy component of our free energy estimates ca ..."
Abstract

Cited by 32 (17 self)
 Add to MetaCart
(Show Context)
We present a technique for approximating the free energy of protein structures using Generalized Belief Propagation (GBP). The accuracy and utility of these estimates are then demonstrated in two different application domains. First, we show that the entropy component of our free energy estimates can be used to distinguish native protein structures from decoys — structures with similar internal energy to that of the native structure, but otherwise incorrect. Our method is able to correctly identify the native fold from among a set of decoys with 87.5 % accuracy over a total of 48 different immunoglobin folds. The remaining 12.5 % of native structures are ranked among the top 4 of all structures. Second, we show that our estimates of ∆∆G upon mutation for three different data sets have linear correlations between 0.640.69 with experimental values and statistically significant pvalues. Together, these results suggests that GBP is an effective means for computing free energy in allatom models of protein structures. GBP is also efficient, taking a few minutes to run on a typical sized protein, further suggesting that GBP may be an attractive alternative to more costly molecular dynamic simulations for some tasks.
The estimation of distributions and the minimum relative entropy principle
 Evolutionary Computation
, 2005
"... Estimation of Distribution Algorithms EDA have been proposed as an extension of genetic algorithms. In this paper the relation of EDA to algorithms developed in statistics, artificial intelligence, and statistical physics is explained. The major design issues are discussed within a general interdisc ..."
Abstract

Cited by 32 (3 self)
 Add to MetaCart
Estimation of Distribution Algorithms EDA have been proposed as an extension of genetic algorithms. In this paper the relation of EDA to algorithms developed in statistics, artificial intelligence, and statistical physics is explained. The major design issues are discussed within a general interdisciplinary framework. It is shown that maximum entropy approximations play a crucial role. All proposed algorithms try to minimize the KullbackLeibler divergence ÃÄ � between the unknown distribution Ô Ü and a class Õ Ü of approximations. The KullbackLeibler divergence is not symmetric. Approximations which suppose that the function to be optimized is additively decomposed (ADF) minimize ÃÄ � Õ�Ô, the methods which learn the approximate model from data minimize ÃÄ � Ô�Õ. This minimization is identical to maximizing the loglikelihood. In the paper three classes of algorithms are discussed. FDA uses the ADF to compute an approximate factorization of the unknown distribution. The factors are marginal distributions, whose values are computed from samples. The BetheKikuchi approach developed in statistical physics uses bivariate or higher order marginals. The values of the marginals are computed from a difficult minimization problem. The third class learns the factorization from the data. We analyze our learning algorithm LFDA in detail. It is shown that learning is faced with two problems: first, to detect the important dependencies between the variables, and second, to create an acyclic Bayesian network of bounded clique size.
An efficient pseudocodeword search algorithm for linear programming decoding of LDPC codes
 IEEE Trans. on Inform. Theory
, 2006
"... Abstract—In linear programming (LP) decoding of a lowdensity paritycheck (LDPC) code one minimizes a linear functional, with coefficients related to loglikelihood ratios, over a relaxation of the polytope spanned by the codewords. In order to quantify LP decoding it is important to study vertexes ..."
Abstract

Cited by 21 (13 self)
 Add to MetaCart
(Show Context)
Abstract—In linear programming (LP) decoding of a lowdensity paritycheck (LDPC) code one minimizes a linear functional, with coefficients related to loglikelihood ratios, over a relaxation of the polytope spanned by the codewords. In order to quantify LP decoding it is important to study vertexes of the relaxed polytope, socalled pseudocodewords. We propose a technique to heuristically create a list of pseudocodewords close to the zero codeword and their distances. Our pseudocodewordsearch algorithm starts by randomly choosing configuration of the noise. The configuration is modified through a discrete number of steps. Each step consists of two substeps: one applies an LP decoder to the noiseconfiguration deriving a pseudocodeword, and then finds configuration of the noise equidistant from the pseudocodeword and the zero codeword. The resulting noise configuration is used as an entry for the next step. The iterations converge rapidly to a pseudocodeword neighboring the zero codeword. Repeated many times, this procedure is characterized by the distribution function of the pseudocodeword effective distance. The efficiency of the procedure is demonstrated on examples of the Tanner code and Margulis codes operating over an additive white Gaussian noise (AWGN) channel. Index Terms—Errorfloor, linear programming decoding, lowdensity paritycheck (LDPC) codes.
Loop corrected belief propagation
 In Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS07
, 2007
"... We propose a method for improving Belief Propagation (BP) that takes into account the influence of loops in the graphical model. The method is a variation on and generalization of the method recently introduced by Montanari and Rizzo [1]. It consists of two steps: (i) standard BP is used to calculat ..."
Abstract

Cited by 17 (5 self)
 Add to MetaCart
(Show Context)
We propose a method for improving Belief Propagation (BP) that takes into account the influence of loops in the graphical model. The method is a variation on and generalization of the method recently introduced by Montanari and Rizzo [1]. It consists of two steps: (i) standard BP is used to calculate cavity distributions for each variable (i.e. probability distributions on the Markov blanket of a variable for a modified graphical model, in which the factors involving that variable have been removed); (ii) all cavity distributions are combined by a messagepassing algorithm to obtain consistent single node marginals. The method is exact if the graphical model contains a single loop. The complexity of the method is exponential in the size of the Markov blankets. The results are very accurate in general: the error is often several orders of magnitude smaller than that of standard BP, as illustrated by numerical experiments.
Learning Multiple Belief Propagation Fixed Points for Real Time Inference
, 2009
"... In the context of inference with expectation constraints, we propose an approach based on the “loopy belief propagation ” algorithm (lpb), as a surrogate to an exact Markov Random Field (mrf) modelling. A prior information composed of correlations among a large set of N variables, is encoded into a ..."
Abstract

Cited by 14 (10 self)
 Add to MetaCart
In the context of inference with expectation constraints, we propose an approach based on the “loopy belief propagation ” algorithm (lpb), as a surrogate to an exact Markov Random Field (mrf) modelling. A prior information composed of correlations among a large set of N variables, is encoded into a graphical model; this encoding is optimized with respect to an approximate decoding procedure (lbp), which is used to infer hidden variables from an observed subset. We focus on the situation where the underlying data have many different statistical components, representing a variety of independent patterns. Considering a single parameter family of models we show how lpb may be used to encode and decode efficiently such information, without solving the NP hard inverse problem yielding the optimal mrf. Contrary to usual practice, we work in the nonconvex Bethe free energy minimization framework, and manage to associate a belief propagation fixed point to each component of the underlying probabilistic mixture. The mean field limit is considered and yields an exact connection with the Hopfield model at finite temperature and steady state, when the number of mixture components is proportional to the number of variables. In addition, we provide an enhanced learning procedure, based on a straightforward multiparameter extension of the model in conjunction with an effective continuous optimization procedure. This is performed using the stochastic search heuristic cmaes and yields a significant improvement with respect to the single parameter basic model. 1
Loop corrections for approximate inference on factor graphs
 Journal of Machine Learning Research
"... We propose a method to improve approximate inference methods by correcting for the influence of loops in the graphical model. The method is a generalization and alternative implementation of a recent idea from Montanari and Rizzo (2005). It is applicable to arbitrary factor graphs, provided that the ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
(Show Context)
We propose a method to improve approximate inference methods by correcting for the influence of loops in the graphical model. The method is a generalization and alternative implementation of a recent idea from Montanari and Rizzo (2005). It is applicable to arbitrary factor graphs, provided that the size of the Markov blankets is not too large. It consists of two steps: (i) an approximate inference method, for example, belief propagation, is used to approximate cavity distributions for each variable (i.e., probability distributions on the Markov blanket of a variable for a modified graphical model in which the factors involving that variable have been removed); (ii) all cavity distributions are improved by a messagepassing algorithm that cancels out approximation errors by imposing certain consistency constraints. This loop correction (LC) method usually gives significantly better results than the original, uncorrected, approximate inference algorithm that is used to estimate the effect of loops. Indeed, we often observe that the loopcorrected error is approximately the square of the error of the uncorrected approximate inference method. In this article, we compare different variants of the loop correction method with other approximate inference methods on a variety of graphical models, including “real world ” networks, and conclude that the LC method generally obtains the most accurate results.
MODELING AND INFERENCE OF SEQUENCESTRUCTURE SPECIFICITY
, 2009
"... In order to evaluate protein sequences for simultaneous satisfaction of evolutionary and physical constraints, this paper develops a graphical model approach integrating sequence information from the evolutionary record of a protein family with structural information based on a molecular mechanics f ..."
Abstract

Cited by 10 (8 self)
 Add to MetaCart
(Show Context)
In order to evaluate protein sequences for simultaneous satisfaction of evolutionary and physical constraints, this paper develops a graphical model approach integrating sequence information from the evolutionary record of a protein family with structural information based on a molecular mechanics force field. Nodes in the graphical model represent choices for the backbone (native vs. nonnative), amino acids (conservation analysis), and sidechain conformations (rotamer library). Edges capture dependence relationships, in both the sequence (correlated mutations) and the structure (direct physical interactions). The sequence and structure components of the model are complementary, in that the structure component may support choices that were not present in the sequence record due to bias and artifacts, while the sequence component may capture other constraints on protein viability, such as permitting an efficient folding pathway. Inferential procedures enable computation of the joint probability of a sequencestructure pair, thereby assessing the quality of the sequence with respect to both the protein family and the specificity of its energetic preference for the native structure against alternate backbone structures. In a case study of WW domains, we show that by using the joint model and evaluating specificity, we obtain better prediction of foldedness of designed proteins (AUC of 0.85) than either a sequenceonly or a structureonly model, and gain insights into how, where, and why the sequence and structure components complement each other.
THE REPLICA SYMMETRIC SOLUTION FOR POTTS MODELS ON dREGULAR GRAPHS
"... Abstract. We provide an explicit formula for the limiting free energy density (logpartition function divided by the number of vertices) for ferromagnetic Potts models on uniformly sparse graph sequences converging locally to the dregular tree for d even, covering all temperature regimes. This form ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
(Show Context)
Abstract. We provide an explicit formula for the limiting free energy density (logpartition function divided by the number of vertices) for ferromagnetic Potts models on uniformly sparse graph sequences converging locally to the dregular tree for d even, covering all temperature regimes. This formula coincides with the Bethe free energy functional evaluated at a suitable fixed point of the belief propagation recursion on the dregular tree, the socalled replica symmetric solution. For uniformly random dregular graphs we further show that the replica symmetric Bethe formula is an upper bound for the asymptotic free energy for any model with permissive interactions. 1.