Results 1  10
of
158
Learning in graphical models
 STATISTICAL SCIENCE
, 2004
"... Statistical applications in fields such as bioinformatics, information retrieval, speech processing, image processing and communications often involve largescale models in which thousands or millions of random variables are linked in complex ways. Graphical models provide a general methodology for ..."
Abstract

Cited by 806 (10 self)
 Add to MetaCart
Statistical applications in fields such as bioinformatics, information retrieval, speech processing, image processing and communications often involve largescale models in which thousands or millions of random variables are linked in complex ways. Graphical models provide a general methodology for approaching these problems, and indeed many of the models developed by researchers in these applied fields are instances of the general graphical model formalism. We review some of the basic ideas underlying graphical models, including the algorithmic ideas that allow graphical models to be deployed in largescale data analysis problems. We also present examples of graphical models in bioinformatics, errorcontrol coding and language processing.
Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms
 IEEE Transactions on Information Theory
, 2005
"... Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems t ..."
Abstract

Cited by 585 (13 self)
 Add to MetaCart
(Show Context)
Important inference problems in statistical physics, computer vision, errorcorrecting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain regionbased free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a “valid ” or “maxentnormal ” approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the “Bethe method, ” the “junction graph method, ” the “cluster variation method, ” and the “region graph method.” Finally, we explain how to tell whether a regionbased approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.
Capacity of MIMO systems with antenna selection
, 2005
"... We consider the capacity of multipleinput multipleoutput systems with reduced complexity. One linkend uses all available antennas, while the other chooses the L out of N antennas that maximize capacity. We derive an upper bound on the capacity that can be expressed sa sthe sum of the logarithms o ..."
Abstract

Cited by 126 (14 self)
 Add to MetaCart
(Show Context)
We consider the capacity of multipleinput multipleoutput systems with reduced complexity. One linkend uses all available antennas, while the other chooses the L out of N antennas that maximize capacity. We derive an upper bound on the capacity that can be expressed sa sthe sum of the logarithms of ordered chisquaredistributed variables. This bound is then evaluated analytically and compared to the results obtained by Monte Carlo simulations. Our results show that the achieved capacity is close to the capacity of a fullcomplexity system provided that L is at least as large as the number of antennas at the other linkend. For example, for L=3, N=8 antennas at the receiver and three antennas at the transmitter, the capacity of the reducedcomplexity scheme is 20 bits/s/Hz compared to 23 bits/s/Hz of a fullcomplexity scheme. We also present a suboptimum antenna subset selection algorithm that has a complexity of N2 compared to eht optimum algorithm with a complexity of (N L).
TreeBased Reparameterization Framework for Analysis of Belief Propagation and Related Algorithms
, 2001
"... We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and ext ..."
Abstract

Cited by 122 (20 self)
 Add to MetaCart
(Show Context)
We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and extensions of belief propagation. Algorithms in this class can be formulated as a sequence of reparameterization updates, each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph (i.e., a tree). The ultimate goal is to obtain an alternative but equivalent factorization using functions that represent (exact or approximate) marginal distributions on cliques of the graph. Our framework highlights an important property of BP and the entire class of reparameterization algorithms: the distribution on the full graph is not changed. The perspective of treebased updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency. We develop interpretations of these results in terms of information geometry. The invariance of the distribution, in conjunction with the fixed point characterization, enables us to derive an exact relation between the exact marginals on an arbitrary graph with cycles, and the approximations provided by belief propagation, and more broadly, any algorithm that minimizes the Bethe free energy. We also develop bounds on this approximation error, which illuminate the conditions that govern their accuracy. Finally, we show how the reparameterization perspective extends naturally to more structured approximations (e.g., Kikuchi and variants [52, 37]) that operate over higher order cliques.
Digital Fountains: A Survey and Look Forward
, 2004
"... We survey constructions and applications of digital fountains, an abstraction of erasure coding for network communication. Digital fountains effectively change the standard paradigm where a user receives an ordered stream of packets to one where a user must simply receive enough packets in order to ..."
Abstract

Cited by 89 (0 self)
 Add to MetaCart
We survey constructions and applications of digital fountains, an abstraction of erasure coding for network communication. Digital fountains effectively change the standard paradigm where a user receives an ordered stream of packets to one where a user must simply receive enough packets in order to obtain the desired data. Obviating the need for ordered data simplifies data delivery, especially when the data is large or is to be distributed to a large number of users. We also examine barriers to the adoption of digital fountains and discuss whether they can be overcome.
Stopping set distribution of LDPC code ensembles
 IEEE TRANS. INFORM. THEORY
, 2005
"... Stopping sets determine the performance of lowdensity paritycheck (LDPC) codes under iterative decoding over erasure channels. We derive several results on the asymptotic behavior of stopping sets in Tannergraph ensembles, including the following. An expression for the normalized average stoppin ..."
Abstract

Cited by 70 (1 self)
 Add to MetaCart
(Show Context)
Stopping sets determine the performance of lowdensity paritycheck (LDPC) codes under iterative decoding over erasure channels. We derive several results on the asymptotic behavior of stopping sets in Tannergraph ensembles, including the following. An expression for the normalized average stopping set distribution, yielding, in particular, a critical fraction of the block length above which codes have exponentially many stopping sets of that size. A relation between the degree distribution and the likely size of the smallest nonempty stopping set, showing that for a I
A New Look at Survey Propagation and its Generalizations
"... We study the survey propagation algorithm [19, 5, 4], which is an iterative technique that appears to be very effective in solving random kSAT problems even with densities close to threshold. We first describe how any SAT formula can be associated with a novel family of Markov random fields (MRFs), ..."
Abstract

Cited by 66 (11 self)
 Add to MetaCart
(Show Context)
We study the survey propagation algorithm [19, 5, 4], which is an iterative technique that appears to be very effective in solving random kSAT problems even with densities close to threshold. We first describe how any SAT formula can be associated with a novel family of Markov random fields (MRFs), parameterized by a real number ρ. We then show that applying belief propagation— a wellknown “messagepassing” technique—to this family of MRFs recovers various algorithms, ranging from pure survey propagation at one extreme (ρ = 1) to standard belief propagation on the uniform distribution over SAT assignments at the other extreme (ρ = 0). Configurations in these MRFs have a natural interpretation as generalized satisfiability assignments, on which a partial order can be defined. We isolate cores as minimal elements in this partial
Tree Consistency and Bounds on the Performance of the MaxProduct Algorithm and Its Generalizations
, 2002
"... Finding the maximum a posteriori (MAP) assignment of a discretestate distribution specified by a graphical model requires solving an integer program. The maxproduct algorithm, also known as the maxplus or minsum algorithm, is an iterative method for (approximately) solving such a problem on gr ..."
Abstract

Cited by 65 (5 self)
 Add to MetaCart
Finding the maximum a posteriori (MAP) assignment of a discretestate distribution specified by a graphical model requires solving an integer program. The maxproduct algorithm, also known as the maxplus or minsum algorithm, is an iterative method for (approximately) solving such a problem on graphs with cycles.
LH*RS  a highavailability scalable distributed data structure
"... (SDDS). An LH*RS file is hash partitioned over the distributed RAM of a multicomputer, e.g., a network of PCs, and supports the unavailability of any of its k ≥ 1 server nodes. The value of k transparently grows with the file to offset the reliability decline. Only the number of the storage nodes p ..."
Abstract

Cited by 59 (11 self)
 Add to MetaCart
(SDDS). An LH*RS file is hash partitioned over the distributed RAM of a multicomputer, e.g., a network of PCs, and supports the unavailability of any of its k ≥ 1 server nodes. The value of k transparently grows with the file to offset the reliability decline. Only the number of the storage nodes potentially limits the file growth. The highavailability management uses a novel parity calculus that we have developed, based on the ReedSalomon erasure correcting coding. The resulting parity storage overhead is about the minimal ever possible. The parity encoding and decoding are faster than for any other candidate coding we are aware of. We present our scheme and its performance analysis, including experiments with a prototype implementation on Wintel PCs. The capabilities of LH*RS offer new perspectives to data intensive applications, including the emerging ones of grids and of P2P computing.
Constructions of LDPC Codes using Ramanujan Graphs and Ideas from Margulis
 in Proc. of the 38th Allerton Conference on Communication, Control, and Computing
, 2000
"... Some twenty years ago G.A. Margulis [8] proposed an algebraic construction of LDPC codes. In this paper we analyze the performance of the codes proposed by Margulis. Mimicking the construction of Margulis we describe a new powerful regular LDPC code whose construction is based on a Ramanujan graph. ..."
Abstract

Cited by 54 (6 self)
 Add to MetaCart
(Show Context)
Some twenty years ago G.A. Margulis [8] proposed an algebraic construction of LDPC codes. In this paper we analyze the performance of the codes proposed by Margulis. Mimicking the construction of Margulis we describe a new powerful regular LDPC code whose construction is based on a Ramanujan graph. 1 Introduction LowDensity ParityCheck (LDPC) codes were introduced by Gallager [3] and they have been the focus of intense research in recent years. Roughly speaking, an LDPC code is a binary linear block code having an m n paritycheck matrix H whose nonzero entries are sparse. LDPC codes are typically described by a bipartite graph. The n leftvertices fv 1 ; : : : ; v n g represent the code symbols and the m rightvertices fc 1 ; : : : ; c m g represent the code constraints. There is an edge between vertex v j on the left and vertex c i on the right whenever the entry h i;j of the matrix H is 1. With this the matrix H represents the adjacency matrix of the bipartite graph. We say th...