by Anant Sahai, Sanjoy Mitter
IEEE Transactions on Information Theory
http://www.eecs.berkeley.edu/~sahai/Papers/anytime.pdf
Add To MetaCart
Abstract:
Our understanding of information in systems has been based on the foundation of memoryless processes. Extensions to stable Markov and auto-regressive processes are classical. Berger proved a source coding theorem for the marginally unstable Wiener process, but the infinitehorizon exponentially unstable case had been open since Gray’s 1970 paper. At the same time, there were no theorems showing what was needed to transport such processes across noisy channels. In this work, we give a fixed rate theorem for the infinite-horizon problem of coding an exponentially unstable process. The fixed rate code has the property that the encoding naturally results in two distinct bitstreams that have qualitatively different QoS requirements for subsequent transport over a noisy medium. In order to show this, we introduce the notion of anytime reliability for sequential communication schemes. Necessary and sufficient conditions are established tightly relating the problem of communicating unstable processes at a target distortion to the problem of reliably communicating two streams of bits across a noisy channel with one stream having a higher anytime reliability than the other. This fundamental need for different QoS for different bits arising from a single source is new in information theory. 1
Citations
|
2739
|
A mathematical theory of communication
– Shannon
- 1948
|
|
771
|
Information Theory and Reliable Communication
– Gallager
- 1968
|
|
420
|
Quantization
– Gray, Neuhoff
- 1998
|
|
149
|
Information-Theoretic Analysis of Information Hiding
– Moulin, O’Sullivan
- 2003
|
|
125
|
Coding theorems for a discrete source with a fidelity criterion
– Shannon
- 1959
|
|
115
|
Rate Distortion Theory
– Berger
- 1971
|
|
95
|
The zero error capacity of a noisy channel
– Shannon
- 1956
|
|
64
|
A general formula for channel capacity
– Verdú, Han
- 1994
|
|
63
|
Control under communication constraints
– Tatikonda, Mitter
- 2004
|
|
60
|
The Gaussian Watermarking Game
– Cohen, Lapidoth
- 2002
|
|
60
|
Information Theory and Communication Networks: An Unconsummated Union
– Ephremides, Hajek
- 1998
|
|
44
|
Successive refinement of information: Characterization of the achievable rates
– Rimoldi
- 1994
|
|
29
|
On universal quantization by randomized uniform/lattice quantizers
– Zamir, Feder
- 1992
|
|
27
|
The source-channel separation theorem revisited
– Vembu, Verdú, et al.
- 1995
|
|
24
|
Convolutional codes II: Maximum likelihood decoding
– Forney
- 1974
|
|
19
|
Coding for Interactive Communication
– Schulman
|
|
17
|
Information rates of autoregressive processes
– Gray
- 1970
|
|
16
|
On random coding error exponents of watermarking systems
– Merhav
- 2000
|
|
16
|
Stochastic Linear Control Over a Communication Channel
– Tatikonda, Sahai, et al.
- 2004
|
|
12
|
Convolutional codes III. Sequential decoding
– Forney
- 1974
|
|
9
|
The necessity and sufficiency of anytime capacity for control over a noisy communication link
– Sahai, Mitter
- 2004
|
|
7
|
Information Rates of Wiener Processes
– Berger
- 1970
|
|
5
|
A coding theorem for a class of unknown channels
– Stiglitz
- 1967
|
|
4
|
On the Rate-Distortion Function for the Nonstationary Autoregressive Process
– Hashimoto, Arimoto
- 1980
|
|
4
|
Any-time” Capacity and A Separation Theorem For Tracking Unstable
– Sahai
- 2000
|
|
4
|
A Fundamental Need for Differentiated “Quality of Service” Over Communication Links: An Information Theoretic Approach
– Sahai, Mitter
- 2000
|
|
2
|
A Variable Rate Source-Coding Theorem for Unstable Scalar
– Sahai
- 2001
|
|
1
|
Rate-distortion theory for ergodic sources with side information
– Leiner, Gray
- 1974
|
|
1
|
Information Flow and Entropy Production in the Kalman-Bucy Filter
– Mitter, Newton
- 2004
|
|
1
|
The sequential rate distortion function” submitted to
– Tatikonda, Mitter
|
