#### DMCA

## Tree-structure expectation propagation for LDPC decoding over the BEC (2013)

Venue: | IEEE Trans. Inf. Theory |

Citations: | 4 - 4 self |

### Citations

12380 |
Elements of Information Theory
- Thomas, Cover
- 1991
(Show Context)
Citation Context ...der the set of all possible probability distributions in a given exponential family that map over the same factor graph. EP minimizes within this family the inclusive Kullback-Leibler (KL) divergence =-=[25]-=- with respect to p(V). In [24], [26], it is shown that BP can be reformulated as EP by considering a discrete family of pdfs ar X iv :1 20 1. 07 15 v3s[ cs .IT ]s13sA ugs20 12 IEEE TRANSACTIONS ON INF... |

8886 |
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- Pearl
- 1988
(Show Context)
Citation Context ...ation (BP) algorithm plays a fundamental role. BP was later redescribed and popularized in the articial intelligence community to perform approximate inference over graphical models, see for instance =-=[2]-=-, [3], [4]. Given a factor graph that represents a joint probability density function (pdf) p(V) of a set of discrete random variables [5], BP estimates the marginal probability function for each vari... |

1787 | Factor graphs and the sum-product algorithm
- Kschischang, Frey, et al.
- 2001
(Show Context)
Citation Context ...en the nodes of the graph. The complexity of this algorithm is linear in the number of nodes [2]. For tree-like graphs, the BP solution is exact, but for graphs with cycles, BP is strictly suboptimal =-=[6]-=-, [7], [8]. Linear block codes can be represented using factor (Tanner) graphs [9], where the factor nodes enforce the parity check This work was partially funded by Spanish government (Ministerio de ... |

748 | Good Error Correcting Codes Based on Very Sparse Matrices
- MacKay
- 1999
(Show Context)
Citation Context ... algorithm plays a fundamental role. BP was later redescribed and popularized in the articial intelligence community to perform approximate inference over graphical models, see for instance [2], [3], =-=[4]-=-. Given a factor graph that represents a joint probability density function (pdf) p(V) of a set of discrete random variables [5], BP estimates the marginal probability function for each variable. It u... |

637 |
Low Density Parity Check Codes
- Gallager
- 1963
(Show Context)
Citation Context ...ximates the TEP improved performance and facilitates its optimization. I. INTRODUCTION Low-density parity-check (LDPC) codes are well knownchannel capacity-approaching (c.a.) linear codes. In his PhD =-=[1]-=-, Gallager proposed LDPC codes along with lineartime practical decoding methods, among which the belief propagation (BP) algorithm plays a fundamental role. BP was later redescribed and popularized in... |

637 | The infinite hidden markov model, in:
- Beal, Ghahramani, et al.
- 2002
(Show Context)
Citation Context .... . . qn(Vn). EP generalizes BP in two ways: first, it is not restricted to discrete random variables. And second, EP naturally formulates to include more versatile approximating factorizations [27], =-=[28]-=-. In this paper, we focus on EP to construct a Markov tree-structure to approximate the original graph. Conditional factors in the tree-structure are able to capture pairwise interactions that single ... |

636 |
A recursive approach to low complexity codes
- Tanner
- 1981
(Show Context)
Citation Context ...r of nodes [2]. For tree-like graphs, the BP solution is exact, but for graphs with cycles, BP is strictly suboptimal [6], [7], [8]. Linear block codes can be represented using factor (Tanner) graphs =-=[9]-=-, where the factor nodes enforce the parity check This work was partially funded by Spanish government (Ministerio de Educación y Ciencia, TEC2009-14504-C02-01,02, Consolider-Ingenio 2010 CSD2008-000... |

500 | Near Shannon limit performance of low density parity check codes,”
- MacKay, Neal
- 1996
(Show Context)
Citation Context ... (BP) algorithm plays a fundamental role. BP was later redescribed and popularized in the articial intelligence community to perform approximate inference over graphical models, see for instance [2], =-=[3]-=-, [4]. Given a factor graph that represents a joint probability density function (pdf) p(V) of a set of discrete random variables [5], BP estimates the marginal probability function for each variable.... |

473 | Generalized belief propagation
- Yedidia, Freeman, et al.
- 2000
(Show Context)
Citation Context ...ity distributions in a given exponential family that map over the same factor graph. EP minimizes within this family the inclusive Kullback-Leibler (KL) divergence [25] with respect to p(V). In [24], =-=[26]-=-, it is shown that BP can be reformulated as EP by considering a discrete family of pdfs ar X iv :1 20 1. 07 15 v3s[ cs .IT ]s13sA ugs20 12 IEEE TRANSACTIONS ON INFORMATION THEORY 2 that factorizes as... |

449 |
Expectation propagation for approximate bayesian inference
- Minka
- 2001
(Show Context)
Citation Context ...termining the dominant terms of the code weight distribution [14], [18]. Precise expressions for the asymptotic bit-MAP and BP error floor are derived in [1], [22], [23]. Expectation propagation (EP) =-=[24]-=- can be understood as a generalization of BP to construct tractable approximations of a joint pdf p(V). Consider the set of all possible probability distributions in a given exponential family that ma... |

431 |
Pattern Recognition and Machine Learning (Information Science and Statistics
- Bishop
- 2006
(Show Context)
Citation Context ...ion of p (V|y) in (2) by a complete disconnected factor graph, i.e. p(V|y) ∝ n∏ i=1 p(yi|Vi) k∏ j=1 Cj(V) ≈ n∏ i=1 q̂i,BP(Vi), (4) where q̂i,BP(Vi) is the BP estimate for the i-th variable [6], [37], =-=[38]-=-. 1In the following, we use lower case letters to denote a particular realization of a random variable or vector, e.g V = v means that V takes the v value. IEEE TRANSACTIONS ON INFORMATION THEORY 3 A.... |

366 | A family of algorithms for approximate Bayesian inference.
- Minka
- 2001
(Show Context)
Citation Context ...ular realization of a random variable or vector, e.g V = v means that V takes the v value. IEEE TRANSACTIONS ON INFORMATION THEORY 3 A. Tree-EP algorithm for LDPC decoding The Tree-EP algorithm [27], =-=[39]-=- improves BP decoding because it approximates the posterior p (V|y) in (2) with a tree (or forest) Markov-structure between the variables, i.e.: q(V) = n∏ i=1 qi(Vi|Vpi), (5) where the set of pairs {(... |

359 | The Generalized Distributive Law,”
- Aji, McEliece
- 2000
(Show Context)
Citation Context ...es of the graph. The complexity of this algorithm is linear in the number of nodes [2]. For tree-like graphs, the BP solution is exact, but for graphs with cycles, BP is strictly suboptimal [6], [7], =-=[8]-=-. Linear block codes can be represented using factor (Tanner) graphs [9], where the factor nodes enforce the parity check This work was partially funded by Spanish government (Ministerio de Educación... |

358 | Codes and decoding on general graphs
- Wiberg
- 1996
(Show Context)
Citation Context ...e nodes of the graph. The complexity of this algorithm is linear in the number of nodes [2]. For tree-like graphs, the BP solution is exact, but for graphs with cycles, BP is strictly suboptimal [6], =-=[7]-=-, [8]. Linear block codes can be represented using factor (Tanner) graphs [9], where the factor nodes enforce the parity check This work was partially funded by Spanish government (Ministerio de Educa... |

358 | Efficient erasure correcting codes
- Luby, Mitzenmacher, et al.
- 2001
(Show Context)
Citation Context ...e formulation, in which the known variable nodes (encoded bits) are removed from the graph after each iteration. The BP, under this interpretation, is referred to as the peeling decoder (PD) [11]. In =-=[16]-=-, the authors investigate the PD limiting performance by describing the expected LDPC graph evolution throughout the decoding process by a set of differential equations. The asymptotic performance for... |

284 | Practical loss-resilient codes
- Luby, Mitzenmacher, et al.
- 1997
(Show Context)
Citation Context ...rest of variables are known. Since the BP solution is restricted to this case, the description of the BP as a peeling-type algorithm is obtained if we do not consider Steps 13-16 in Algorithm 2 [16], =-=[43]-=-. In this sense, the TEP decoder emerges as an improved PD. Besides, we claim that the complexity of both decoders is of the same order, i.e. O(n). We intentionally leave to Section IV-E a detailed an... |

239 | Channel coding rate in the finite blocklength regime
- Polyanskiy, Poor, et al.
- 2010
(Show Context)
Citation Context ...ed to decode some finite-length LDPC ensembles over the BEC. The gain of the proposed algorithm for practical codes can be analyzed from a different perspective based on the work by Polyanskiy et al. =-=[49]-=-. They present bounds on the maximum achievable coding rate for binary memoryless channels in the finite-length regime. These bounds can be regarded as the extension of the Shannon coding rate limit w... |

208 | Urbanke, “The capacity of low-density parity check codes under message-passing decoding
- Richardson, L
- 2001
(Show Context)
Citation Context ...th the code length n. For large block lengths, a channel decoder based on BP achieves an excellent performance, close to the bitwise maximum a posteriori (bit-MAP) decoding, in certain scenarios [3], =-=[10]-=-. Nevertheless, the bit-MAP solution can only be achieved when the code length, code density and computational complexity go to infinity [11], [12], [13]. The analysis of the BP for LDPC decoding over... |

197 | An introduction to factor graphs
- Loeliger
- 2004
(Show Context)
Citation Context ...approximate inference over graphical models, see for instance [2], [3], [4]. Given a factor graph that represents a joint probability density function (pdf) p(V) of a set of discrete random variables =-=[5]-=-, BP estimates the marginal probability function for each variable. It uses a local message-passing algorithm between the nodes of the graph. The complexity of this algorithm is linear in the number o... |

184 | Extrinsic information transfer functions: Model and erasure channel properties - Ashikhmin, Kramer, et al. - 2004 |

172 | Propagation of probabilities, means and variances in mixed graphical association models. - Lauritzen - 1992 |

159 |
Differential equations for random processes and random graphs. The annals of applied probability,
- Wormald
- 1995
(Show Context)
Citation Context ...s the solution of a set of differential equations and characterized the typical deviation from it. Their analysis is based on a result on the evolution of (martingale) Markov processes due to Wormald =-=[47]-=-. In this section, we first introduce the Wormald’s theorem and then particularize it to compute the expected evolution of the residual graphs for the TEP, which is used in the following to evaluate t... |

137 | Threshold saturation via spatial coupling: why convolutional LDPC ensembles perform so well over the BEC,” e-print,
- Kudekar, Richardson, et al.
- 2010
(Show Context)
Citation Context ... (bit-MAP) decoding, in certain scenarios [3], [10]. Nevertheless, the bit-MAP solution can only be achieved when the code length, code density and computational complexity go to infinity [11], [12], =-=[13]-=-. The analysis of the BP for LDPC decoding over independent and identically distributed channels is detailed in [14], [15], in which the limiting performance and code optimization are addressed. For t... |

110 |
Error Correction Coding: Mathematical Methods and Algorithms.
- Moon
- 2005
(Show Context)
Citation Context ...Assume that an unknown codeword is transmitted through a discrete memoryless channel [25] and let y ∈ A(y) be the observed channel output, where A(y) is the channel output alphabet. A bit-MAP decoder =-=[35]-=- minimizes the bit error rate (BER) by estimating the transmitted vector v̂ = [v̂1, v̂2, . . . , v̂n] as follows1: v̂u = arg max v∈{0,1} p(Vu = v|y) = arg max v∈{0,1} ∑ V∈C:Vu=v p (V|y) = arg max v∈{0... |

91 | Randomized algorithms for the loop cutset problem.
- Becker, Bar-Yehuda, et al.
- 2000
(Show Context)
Citation Context ...(20) is solved at linear cost by message-passing. For the latter, where the graph is not completely cycle-free, we can compute the pairwise marginals using Pearl’s cutset conditioning algorithm [27], =-=[42]-=-. Pearl’s algorithm proceeds by breaking each cycle assuming a set of the variables involved as known, e.g. Vo in Fig. 1(b). Then, the marginals of the remaining variables can be computed at low-cost ... |

88 |
Finite length analysis of low-density parity-check codes,”
- Di, Proietti, et al.
- 2002
(Show Context)
Citation Context ...e code length, code density and computational complexity go to infinity [11], [12], [13]. The analysis of the BP for LDPC decoding over independent and identically distributed channels is detailed in =-=[14]-=-, [15], in which the limiting performance and code optimization are addressed. For the binary erasure channel (BEC), the BP decoder presents an alternative formulation, in which the known variable nod... |

87 | Capacity-achieving sequences for the erasure channel
- Oswald, Shokrollahi
- 2002
(Show Context)
Citation Context ... length, code density and computational complexity go to infinity [11], [12], [13]. The analysis of the BP for LDPC decoding over independent and identically distributed channels is detailed in [14], =-=[15]-=-, in which the limiting performance and code optimization are addressed. For the binary erasure channel (BEC), the BP decoder presents an alternative formulation, in which the known variable nodes (en... |

70 | Stopping Set Distribution of LDPC Code Ensembles
- Orlitsky, Viswanathan, et al.
- 2005
(Show Context)
Citation Context ... the error floor is addressed by determining the dominant terms of the code weight distribution [14], [18]. Precise expressions for the asymptotic bit-MAP and BP error floor are derived in [1], [22], =-=[23]-=-. Expectation propagation (EP) [24] can be understood as a generalization of BP to construct tractable approximations of a joint pdf p(V). Consider the set of all possible probability distributions in... |

54 |
Tree-structured approximations by expectation propagation.
- Minka, Qi
- 2004
(Show Context)
Citation Context ...2(V2) . . . qn(Vn). EP generalizes BP in two ways: first, it is not restricted to discrete random variables. And second, EP naturally formulates to include more versatile approximating factorizations =-=[27]-=-, [28]. In this paper, we focus on EP to construct a Markov tree-structure to approximate the original graph. Conditional factors in the tree-structure are able to capture pairwise interactions that s... |

44 | Which codes have cycle-free tanner graphs
- Etzion, Trachtenberg, et al.
- 1999
(Show Context)
Citation Context ...ryless and that all codewords are equally probable p(V) = 1[V ∈ C] 2nr . (3) For most LDPC codes of interest, the factor graph associated to the product p (y|V) p(V) in (2) yields a graph with cycles =-=[36]-=-. Hence, the exact computation of the marginals of p(Vu = v|y) grows exponentially with the number of coded bits [6]. Belief propagation [1], [2], [3] is nowadays the standard algorithm to efficiently... |

41 | Maxwell construction: The hidden bridge between iterative and maximum a posteriori decoding - Measson, Montanari, et al. - 2008 |

39 | Finite-Length Scaling for Iteratively Decoded LDPC Ensembles
- Amraoui, Montanari, et al.
- 2003
(Show Context)
Citation Context ...allenging if the degree of irregularity or block length increases [11]. Alternatively, we can separate the contributions to the error rate of large-size errors, which dominate in the waterfall region =-=[19]-=-, from small failures, which cause error floors [14]. Scaling laws (SLs) were proposed in [19], [20] to accurately predict the BP performance in the waterfall region. For the BEC, they are based on th... |

38 |
Weight distribution of lowdensity parity-check codes
- Di, Richardson, et al.
- 2006
(Show Context)
Citation Context ...sis of the error floor is addressed by determining the dominant terms of the code weight distribution [14], [18]. Precise expressions for the asymptotic bit-MAP and BP error floor are derived in [1], =-=[22]-=-, [23]. Expectation propagation (EP) [24] can be understood as a generalization of BP to construct tractable approximations of a joint pdf p(V). Consider the set of all possible probability distributi... |

30 | On decoding of low-density parity-check codes over the binary erasure channel,”
- Pishro-Nik, Fekri
- 2004
(Show Context)
Citation Context ...he same order than BP, i.e. linear in the number of variables, unlike other techniques proposed to improve BP at a higher computational cost. For instance, we can mention variable guessing algorithms =-=[31]-=-, the Maxwell decoder [12] and pivoting algorithms for efficient Gaussian elimination [32], [33], [34], whose complexity is not linear unless we impose additional restrictions that may alter/compromis... |

30 | Accumulate-repeat-accumulate codes,”
- Abbasfar, Divsalar, et al.
- 2007
(Show Context)
Citation Context ...e the TEP has removed P3 and V2. B. Connection to previous works A similar procedure for removing degree-two check nodes was considered in the analysis of accumulate-repeataccumulate (ARA) LDPC codes =-=[44]-=-, [11]. ARA codes were proposed to achieve channel capacity under BP decoding at bounded complexity. Roughly speaking, ARA codes are formed by the concatenation of an accumulate binary encoder, an irr... |

23 | An efficient maximum-likelihood decoding of ldpc codes over the binary erasure channel.
- Burshtein, Miller
- 2004
(Show Context)
Citation Context ...posed to improve BP at a higher computational cost. For instance, we can mention variable guessing algorithms [31], the Maxwell decoder [12] and pivoting algorithms for efficient Gaussian elimination =-=[32]-=-, [33], [34], whose complexity is not linear unless we impose additional restrictions that may alter/compromise their performance, such as bounding the number of guessed variables or pivots. The rest ... |

22 |
Accumulate-repeat-accumulate codes: Capacityachieving ensembles of systematic codes for the erasure channel with bounded complexity
- Pfister, Sason
- 2007
(Show Context)
Citation Context ... under BP decoding at bounded complexity. Roughly speaking, ARA codes are formed by the concatenation of an accumulate binary encoder, an irregular LDPC code and another accumulate binary encoder. In =-=[45]-=-, [46], Pfister and Sason showed that ARA codes can be described for BEC as an equivalent irregular LDPC ensemble and hence they were able to compute the ARA BP threshold using standard techniques [10... |

11 |
Finite-Length Scaling of Irregular LDPC ensembles
- Amraoui, Montanari, et al.
(Show Context)
Citation Context ...te that any realization xBPr1 (τ, n) of such process represents a successful decoding as long as it is positive for any τ ∈ [0, n/E). The process XBPr1 (τ, n) presents some important properties [19], =-=[20]-=-: 1) E[XBPr1 (τ, n)] closely follows r BP 1 (τ) in (86). For moderately-sized codes the mean of the process is essentially independent of n. 2) The variance is of order O(n−1). We denote it as δBPr1,r... |

10 |
Finite-length analysis of LDPC codes with large left degrees
- Zhang, Orlitsky
(Show Context)
Citation Context ... decoding performance in the finitelength regime is based on the evaluation of the presence of stopping sets (SSs) in the LDPC graph [14], which can severely degrade the decoder performance. In [14], =-=[18]-=-, the authors provide tools to compute the exact BP average performance. However, this task becomes computationally challenging if the degree of irregularity or block length increases [11]. Alternativ... |

10 |
Tractable Inference for Complex Stochastic
- Boyen, Koller
- 1998
(Show Context)
Citation Context ...), which is the one used for mean-field approximations, see [39] for a discussion. where V ∼ Vi denotes all the variables in V except those in Vi. Proof: The proof of this lemma can be found in [40], =-=[41]-=-. Lemma 1 can be directly applied to the problem in (6) and the optimum Markov-tree q̂(V) is such that q̂i(Vi|Vpi) = ∑ V∼{Vi,Vpi} p(V|y) ∑ V∼Vpi p(V|y) = p(Vi|Vpi ,y) (11) for i = 1, . . . , n and q... |

8 |
Pivoting algorithms for maximum likelihood decoding of LDPC codes over erasure channels
- Liva, Matuz, et al.
(Show Context)
Citation Context ...to improve BP at a higher computational cost. For instance, we can mention variable guessing algorithms [31], the Maxwell decoder [12] and pivoting algorithms for efficient Gaussian elimination [32], =-=[33]-=-, [34], whose complexity is not linear unless we impose additional restrictions that may alter/compromise their performance, such as bounding the number of guessed variables or pivots. The rest of the... |

7 | Tree-structured expectation propagation for decoding finite-length LDPC codes
- Olmos, Murillo-Fuentes, et al.
- 2011
(Show Context)
Citation Context ...w that the Tree-EP can be reinterpreted as a peeling-type algorithm that formulates as an improved PD. We refer to this simplified algorithm as the TEP decoder. The TEP decoder was presented in [29], =-=[30]-=-, where we empirically observed a noticeable gain in performance compared to BP for both regular and irregular LDPC codes. We now explain, analyze and predict this gain in performance for any LDPC cod... |

6 | A generalization of the finitelength scaling approach beyond the
- Ezri, Montanari, et al.
- 2007
(Show Context)
Citation Context ... = Q √n(BP − ) αBP + γTEP√ n δBPr1,r1(τ ∗) , (96) where P TEPW (C, ) is the TEP block error rate for the code C ∈ LDPC[λ(x), ρ(x), n] and we have used the values computed in [19], [21], =-=[50]-=-, [51] for the BP (namely αBP and δBPr1,r1(τ ∗)) for the TEP performance. These values measure the mean and variance of the degree-one check nodes and mainly depend on the DD ensemble and the channel ... |

5 | Tree-structure expectation propagation for decoding LDPC codes over binary erasure channels
- Olmos, Murillo-Fuentes, et al.
- 2010
(Show Context)
Citation Context ...we show that the Tree-EP can be reinterpreted as a peeling-type algorithm that formulates as an improved PD. We refer to this simplified algorithm as the TEP decoder. The TEP decoder was presented in =-=[29]-=-, [30], where we empirically observed a noticeable gain in performance compared to BP for both regular and irregular LDPC codes. We now explain, analyze and predict this gain in performance for any LD... |

5 | The Slope Scaling Parameter for General Channels, Decoders, and Ensembles
- Ezri, Montanari, et al.
- 2008
(Show Context)
Citation Context ... = Q √n(BP − ) αBP + γTEP√ n δBPr1,r1(τ ∗) , (96) where P TEPW (C, ) is the TEP block error rate for the code C ∈ LDPC[λ(x), ρ(x), n] and we have used the values computed in [19], [21], [50], =-=[51]-=- for the BP (namely αBP and δBPr1,r1(τ ∗)) for the TEP performance. These values measure the mean and variance of the degree-one check nodes and mainly depend on the DD ensemble and the channel disper... |

5 | Average stopping set weight distributions of redundant random ensembles - Wadayama - 2008 |

3 |
An efficient algorithm for ML decoding of raptor codes over the binary erasure channel
- Kim, Lee, et al.
- 2008
(Show Context)
Citation Context ...rove BP at a higher computational cost. For instance, we can mention variable guessing algorithms [31], the Maxwell decoder [12] and pivoting algorithms for efficient Gaussian elimination [32], [33], =-=[34]-=-, whose complexity is not linear unless we impose additional restrictions that may alter/compromise their performance, such as bounding the number of guessed variables or pivots. The rest of the paper... |

3 | Asymptotic rate versus design rate - Méasson, Montanari, et al. - 2007 |

2 |
Analytical solution of covariance evolution for irregular LDPC codes,” e-prints
- Takayuki, Kasai, et al.
- 2010
(Show Context)
Citation Context ...rmance in the waterfall region. For the BEC, they are based on the PD covariance evolution for a given graph as a function of the code length. Covariance evolution was solved for any LDPC ensemble in =-=[21]-=-. On the other hand, the analysis of the error floor is addressed by determining the dominant terms of the code weight distribution [14], [18]. Precise expressions for the asymptotic bit-MAP and BP er... |

2 | Finite-length analysis of a capacity-achieving ensemble for the binary erasure channel
- Pfister
- 2005
(Show Context)
Citation Context ... BP decoding at bounded complexity. Roughly speaking, ARA codes are formed by the concatenation of an accumulate binary encoder, an irregular LDPC code and another accumulate binary encoder. In [45], =-=[46]-=-, Pfister and Sason showed that ARA codes can be described for BEC as an equivalent irregular LDPC ensemble and hence they were able to compute the ARA BP threshold using standard techniques [10]. To ... |

1 | On the stopping distance and the stopping redundancy of codes,” Information Theory - Schwartz, Vardy - 2006 |