Results 1  10
of
500
Good ErrorCorrecting Codes based on Very Sparse Matrices
, 1999
"... We study two families of errorcorrecting codes defined in terms of very sparse matrices. "MN" (MacKayNeal) codes are recently invented, and "Gallager codes" were first investigated in 1962, but appear to have been largely forgotten, in spite of their excellent properties. The ..."
Abstract

Cited by 750 (23 self)
 Add to MetaCart
We study two families of errorcorrecting codes defined in terms of very sparse matrices. "MN" (MacKayNeal) codes are recently invented, and "Gallager codes" were first investigated in 1962, but appear to have been largely forgotten, in spite of their excellent properties. The decoding of both codes can be tackled with a practical sumproduct algorithm. We prove that these codes are "very good," in that sequences of codes exist which, when optimally decoded, achieve information rates up to the Shannon limit. This result holds not only for the binarysymmetric channel but also for any channel with symmetric stationary ergodic noise. We give experimental results for binarysymmetric channels and Gaussian channels demonstrating that practical performance substantially better than that of standard convolutional and concatenated codes can be achieved; indeed, the performance of Gallager codes is almost as close to the Shannon limit as that of turbo codes.
Turbo decoding as an instance of Pearl’s belief propagation algorithm
 IEEE Journal on Selected Areas in Communications
, 1998
"... Abstract—In this paper, we will describe the close connection between the now celebrated iterative turbo decoding algorithm of Berrou et al. and an algorithm that has been well known in the artificial intelligence community for a decade, but which is relatively unknown to information theorists: Pear ..."
Abstract

Cited by 404 (16 self)
 Add to MetaCart
(Show Context)
Abstract—In this paper, we will describe the close connection between the now celebrated iterative turbo decoding algorithm of Berrou et al. and an algorithm that has been well known in the artificial intelligence community for a decade, but which is relatively unknown to information theorists: Pearl’s belief propagation algorithm. We shall see that if Pearl’s algorithm is applied to the “belief network ” of a parallel concatenation of two or more codes, the turbo decoding algorithm immediately results. Unfortunately, however, this belief diagram has loops, and Pearl only proved that his algorithm works when there are no loops, so an explanation of the excellent experimental performance of turbo decoding is still lacking. However, we shall also show that Pearl’s algorithm can be used to routinely derive previously known iterative, but suboptimal, decoding algorithms for a number of other errorcontrol systems, including Gallager’s
Efficient erasure correcting codes
 IEEE TRANSACTIONS ON INFORMATION THEORY
, 2001
"... We introduce a simple erasure recovery algorithm for codes derived from cascades of sparse bipartite graphs and analyze the algorithm by analyzing a corresponding discretetime random process. As a result, we obtain a simple criterion involving the fractions of nodes of different degrees on both si ..."
Abstract

Cited by 360 (26 self)
 Add to MetaCart
(Show Context)
We introduce a simple erasure recovery algorithm for codes derived from cascades of sparse bipartite graphs and analyze the algorithm by analyzing a corresponding discretetime random process. As a result, we obtain a simple criterion involving the fractions of nodes of different degrees on both sides of the graph which is necessary and sufficient for the decoding process to finish successfully with high probability. By carefully designing these graphs we can construct for any given rate and any given real number a family of linear codes of rate which can be encoded in time proportional to ��@I A times their block length. Furthermore, a codeword can be recovered with high probability from a portion of its entries of length @IC A or more. The recovery algorithm also runs in time proportional to ��@I A. Our algorithms have been implemented and work well in practice; various implementation issues are discussed.
On the design of lowdensity paritycheck codes within 0.0045 dB of the Shannon limit
 IEEE COMMUNICATIONS LETTERS
, 2001
"... We develop improved algorithms to construct good lowdensity paritycheck codes that approach the Shannon limit very closely. For rate 1/2, the best code found has a threshold within 0.0045 dB of the Shannon limit of the binaryinput additive white Gaussian noise channel. Simulation results with a ..."
Abstract

Cited by 306 (6 self)
 Add to MetaCart
(Show Context)
We develop improved algorithms to construct good lowdensity paritycheck codes that approach the Shannon limit very closely. For rate 1/2, the best code found has a threshold within 0.0045 dB of the Shannon limit of the binaryinput additive white Gaussian noise channel. Simulation results with a somewhat simpler code show that we can achieve within 0.04 dB of the Shannon limit at a bit error rate of 10 T using a block length of 10 U.
Analysis of sumproduct decoding of lowdensity paritycheck codes using a Gaussian approximation
 IEEE TRANS. INFORM. THEORY
, 2001
"... Density evolution is an algorithm for computing the capacity of lowdensity paritycheck (LDPC) codes under messagepassing decoding. For memoryless binaryinput continuousoutput additive white Gaussian noise (AWGN) channels and sumproduct decoders, we use a Gaussian approximation for message densi ..."
Abstract

Cited by 244 (2 self)
 Add to MetaCart
(Show Context)
Density evolution is an algorithm for computing the capacity of lowdensity paritycheck (LDPC) codes under messagepassing decoding. For memoryless binaryinput continuousoutput additive white Gaussian noise (AWGN) channels and sumproduct decoders, we use a Gaussian approximation for message densities under density evolution to simplify the analysis of the decoding algorithm. We convert the infinitedimensional problem of iteratively calculating message densities, which is needed to find the exact threshold, to a onedimensional problem of updating means of Gaussian densities. This simplification not only allows us to calculate the threshold quickly and to understand the behavior of the decoder better, but also makes it easier to design good irregular LDPC codes for AWGN channels. For various regular LDPC codes we have examined, thresholds can be estimated within 0.1 dB of the exact value. For rates between 0.5 and 0.9, codes designed using the Gaussian approximation perform within 0.02 dB of the best performing codes found so far by using density evolution when the maximum variable degree is IH. We show that by using the Gaussian approximation, we can visualize the sumproduct decoding algorithm. We also show that the optimization of degree distributions can be understood and done graphically using the visualization.
Improved lowdensity paritycheck codes using irregular graphs
 IEEE Trans. Inform. Theory
, 2001
"... Abstract—We construct new families of errorcorrecting codes based on Gallager’s lowdensity paritycheck codes. We improve on Gallager’s results by introducing irregular paritycheck matrices and a new rigorous analysis of harddecision decoding of these codes. We also provide efficient methods for ..."
Abstract

Cited by 223 (15 self)
 Add to MetaCart
Abstract—We construct new families of errorcorrecting codes based on Gallager’s lowdensity paritycheck codes. We improve on Gallager’s results by introducing irregular paritycheck matrices and a new rigorous analysis of harddecision decoding of these codes. We also provide efficient methods for finding good irregular structures for such decoding algorithms. Our rigorous analysis based on martingales, our methodology for constructing good irregular codes, and the demonstration that irregular structure improves performance constitute key points of our contribution. We also consider irregular codes under belief propagation. We report the results of experiments testing the efficacy of irregular codes on both binarysymmetric and Gaussian channels. For example, using belief propagation, for rate I R codes on 16 000 bits over a binarysymmetric channel, previous lowdensity paritycheck codes can correct up to approximately 16 % errors, while our codes correct over 17%. In some cases our results come very close to reported results for turbo codes, suggesting that variations of irregular low density paritycheck codes may be able to match or beat turbo code performance. Index Terms—Belief propagation, concentration theorem, Gallager codes, irregular codes, lowdensity paritycheck codes.
Lowdensity paritycheck codes based on finite geometries: A rediscovery and new results
 IEEE Trans. Inform. Theory
, 2001
"... This paper presents a geometric approach to the construction of lowdensity paritycheck (LDPC) codes. Four classes of LDPC codes are constructed based on the lines and points of Euclidean and projective geometries over finite fields. Codes of these four classes have good minimum distances and thei ..."
Abstract

Cited by 186 (8 self)
 Add to MetaCart
This paper presents a geometric approach to the construction of lowdensity paritycheck (LDPC) codes. Four classes of LDPC codes are constructed based on the lines and points of Euclidean and projective geometries over finite fields. Codes of these four classes have good minimum distances and their Tanner graphs have girth T. Finitegeometry LDPC codes can be decoded in various ways, ranging from low to high decoding complexity and from reasonably good to very good performance. They perform very well with iterative decoding. Furthermore, they can be put in either cyclic or quasicyclic form. Consequently, their encoding can be achieved in linear time and implemented with simple feedback shift registers. This advantage is not shared by other LDPC codes in general and is important in practice. Finitegeometry LDPC codes can be extended and shortened in various ways to obtain other good LDPC codes. Several techniques of extension and shortening are presented. Long extended finitegeometry LDPC codes have been constructed and they achieve a performance only a few tenths of a decibel away from the Shannon theoretical limit with iterative decoding.
Efficient Encoding of LowDensity ParityCheck Codes
, 2001
"... Lowdensity paritycheck (LDPC) codes can be considered serious competitors to turbo codes in terms of performance and complexity and they are based on a similar philosophy: constrained random code ensembles and iterative decoding algorithms. In this paper, we consider the encoding problem for LDPC ..."
Abstract

Cited by 183 (3 self)
 Add to MetaCart
Lowdensity paritycheck (LDPC) codes can be considered serious competitors to turbo codes in terms of performance and complexity and they are based on a similar philosophy: constrained random code ensembles and iterative decoding algorithms. In this paper, we consider the encoding problem for LDPC codes. More generally, we consider the encoding problem for codes specified by sparse paritycheck matrices. We show how to exploit the sparseness of the paritycheck matrix to obtain efficient encoders. For the @Q TAregular LDPC code, for example, the complexity of encoding is essentially quadratic in the block length. However, we show that the associated coefficient can be made quite small, so that encoding codes even of length IHH HHH is still quite practical. More importantly, we will show that “optimized” codes actually admit linear time encoding.
A Revolution: Belief Propagation in Graphs With Cycles
 In Neural Information Processing Systems
, 1997
"... Until recently, artificial intelligence researchers have frowned upon the application of probability propagation in Bayesian belief networks that have cycles. The probability propagation algorithm is only exact in networks that are cyclefree. However, it has recently been discovered that the tw ..."
Abstract

Cited by 145 (4 self)
 Add to MetaCart
Until recently, artificial intelligence researchers have frowned upon the application of probability propagation in Bayesian belief networks that have cycles. The probability propagation algorithm is only exact in networks that are cyclefree. However, it has recently been discovered that the two best errorcorrecting decoding algorithms are actually performing probability propagation in belief networks with cycles.
Iterative Decoding of Compound Codes by Probability Propagation in Graphical Models
 IEEE J. Sel. Areas Comm
, 1998
"... Abstract—We present a unified graphical model framework for describing compound codes and deriving iterative decoding algorithms. After reviewing a variety of graphical models (Markov random fields, Tanner graphs, and Bayesian networks), we derive a general distributed marginalization algorithm for ..."
Abstract

Cited by 139 (12 self)
 Add to MetaCart
Abstract—We present a unified graphical model framework for describing compound codes and deriving iterative decoding algorithms. After reviewing a variety of graphical models (Markov random fields, Tanner graphs, and Bayesian networks), we derive a general distributed marginalization algorithm for functions described by factor graphs. From this general algorithm, Pearl’s belief propagation algorithm is easily derived as a special case. We point out that recently developed iterative decoding algorithms for various codes, including “turbo decoding ” of parallelconcatenated convolutional codes, may be viewed as probability propagation in a graphical model of the code. We focus on Bayesian network descriptions of codes, which give a natural input/state/output/channel description of a code and channel, and we indicate how iterative decoders can be developed for paralleland serially concatenated coding systems, product codes, and lowdensity paritycheck codes. Index Terms — Concatenated coding, decoding, graph theory, iterative methods, product codes.