Results 1 
3 of
3
The Information Lost in Erasures
, 2008
"... We consider sources and channels with memory observed through erasure channels. In particular, we examine the impact of sporadic erasures on the fundamental limits of lossless data compression, lossy data compression, channel coding, and denoising. We define the erasure entropy of a collection of ra ..."
Abstract

Cited by 19 (3 self)
 Add to MetaCart
(Show Context)
We consider sources and channels with memory observed through erasure channels. In particular, we examine the impact of sporadic erasures on the fundamental limits of lossless data compression, lossy data compression, channel coding, and denoising. We define the erasure entropy of a collection of random variables as the sum of entropies of the individual variables conditioned on all the rest. The erasure entropy measures the information content carried by each symbol knowing its context. The erasure entropy rate is shown to be the minimal amount of bits per erasure required to recover the lost information in the limit of small erasure probability. When we allow recovery of the erased symbols within a prescribed degree of distortion, the fundamental tradeoff is described by the erasure rate–distortion function which we characterize. We show that in the regime of sporadic erasures, knowledge at the encoder of the erasure locations does not lower the rate required to achieve a given distortion. When no additional encoded information is available, the erased information is reconstructed solely on the basis of its context by a denoiser. Connections between erasure entropy and discrete denoising are developed. The decrease of the capacity of channels with memory due to sporadic memoryless erasures is also characterized in wide generality.
Erasure entropy
 In Proc. of ISIT
, 2006
"... Abstract—We define the erasure entropy of a collection of random variables as the sum of entropies of the individual variables conditioned on all the rest. The erasure entropy rate of a source is defined as the limit of the normalized erasure entropy. The erasure entropy measures the information con ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
(Show Context)
Abstract—We define the erasure entropy of a collection of random variables as the sum of entropies of the individual variables conditioned on all the rest. The erasure entropy rate of a source is defined as the limit of the normalized erasure entropy. The erasure entropy measures the information content carried by each symbol knowing its context. In the setup of a source observed through an erasure channel, we offer an operational characterization of erasure entropy rate as the minimal amount of bits per erasure required to recover the erased information in the limit of small erasure probability. When we allow recovery of the erased symbols within a prescribed degree of distortion, the fundamental tradeoff is described by the erasure ratedistortion function which we characterize. When no additional encoded information is available, the erased information is reconstructed solely on the basis of its context by a denoiser. Connections between erasure entropy and discrete denoising are also explored.
Universal Lossless Compression of Erased Symbols
"... Abstract—A source X goes through an erasure channel whose output is Z. The goal is to compress losslessly X when the compressor knows X and Z and the decompressor knows Z. Wepropose a universal algorithm based on contexttree weighting (CTW), parameterized by a memorylength parameter `. We show tha ..."
Abstract
 Add to MetaCart
Abstract—A source X goes through an erasure channel whose output is Z. The goal is to compress losslessly X when the compressor knows X and Z and the decompressor knows Z. Wepropose a universal algorithm based on contexttree weighting (CTW), parameterized by a memorylength parameter `. We show that if the erasure channel is stationary and memoryless, and X is stationary and ergodic, then the proposed algorithm achieves a compression rate of H(X0jX 01 0 `;Z`) bits per erasure. Index Terms—Contexttree weighting, discrete memoryless erasure channel, entropy, erasure entropy, side information, universal lossless compression.