R. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....and the decoding procedures, which assist each other in our joint estimation schemes. IV. CHANNEL CAPACITY A. Simplified Finite State Markov Channel (FSMC) Model For Clarke s flat Rayleigh channel of (1) where the process is stationary and ergodic, the definition of the capacity in Gallager [23] applies (20) where and denote sequences of channel inputs and outputs, respectively. Here we are interested in the constrained capacity for inputs from a finite uniform constellation, such as the 4 PSK we use. If the decoding delay is constrained to be small enough relative to the decorrelation ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....function. It can be observed that this DMC is weakly symmetric in the sense that its transition probability matrix can be partitioned (along its columns) into symmetric arrays where a symmetric array is de ned as an array whose rows are permutations of each other, and so are the columns [32]; therefore, its capacity is achieved by a uniform input distribution. For each channel SNR, the quantization step size of the q bit demodulator is chosen such that the capacity of this DMC is maximized. 3.6 DBMC Model for Turbo Coded AWGN and Rayleigh Channels Besides the simple DMC model ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....digital recording has been addressed by several authors, for example, 193] 88] 87] and [172] The capacity of an average input powerconstrained, discrete time, memoryless channel with additive, independent and identically distributed (i.i.d. Gaussian noise is given by the well known formula [74] where is the noise variance and is the average inputpower constraint. This result is the discrete time equivalent to Shannon s formula (2) via the sampling theorem. Smith [180] showed that the capacity of an amplitude constrained, discrete time, memoryless Gaussian channel is achieved by a ....

R. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

.... on the set with the property, I # I, # # I 1(# I 2(x ) # I #I 2(x ) 9) The channel is called a Regular channel, if the probability p(x ) only depends on i (x ) It can be verified easily that a Regular channels is always Symmetric in sense of Gallager [15], where in [15] the symmetry condition only involves the channel symbols and not the underlying labelling. For a recent introduction to the Regular channels refer to [16] The authors would like to thank G. D. Forney for his invaluable comments on an earlier version of this article, including ....

.... with the property, I # I, # # I 1(# I 2(x ) # I #I 2(x ) 9) The channel is called a Regular channel, if the probability p(x ) only depends on i (x ) It can be verified easily that a Regular channels is always Symmetric in sense of Gallager [15] where in [15] the symmetry condition only involves the channel symbols and not the underlying labelling. For a recent introduction to the Regular channels refer to [16] The authors would like to thank G. D. Forney for his invaluable comments on an earlier version of this article, including pointing out ....

R. G. Gallager, Information Theory and Reliable Communication, New York: Wiley, 1968.

....a coding scheme which, subject to certain performance criteria, will best compress the given source. Once the relevant source parameters have been identified, the problem reduces to one of minimum redundancy coding. This phase of the problem has received extensive treatment in the literature [l] [7]. When no a priori knowledge of the source characteristics is available, and if statistical tests are either impossible or unreliable, the problem of data compression becomes considerably more complicated. In order to overcome these difficulties one must resort to universal coding schemes whereby ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....but broadband in the sense that the SNR per degree of freedom is very small , i.e. we are power limited as opposed to bandwidth limited. That the bandwidth is small compared to the carrier frequency is the reason why we can define the coherence time only with respect to the carrier frequency in [2]. B. Capacity via Frequency Shift Keying This section is devoted to proving the following theorem. The proof is based on [2, Sec. 8.6] which proves the analogous result for a Rayleigh fading channel, except that we use a threshold decoding rule which allows us to prove a more general result ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....C. Optimal Power and Rate Allocation We now consider the problem of determining for each fading state . Note that Lemma 3. 10 can be viewed as a multiaccess generalization of the Lagrangian formulation for the problem of allocating powers over a set of parallel single user Gaussian channels [6]. The solution to the optimization problem in the single user setting is given by the classic water filling construction. Here we will provide a solution in the multiaccess setting. Again we make use of the polymatroid structure and the solution will have a greedy flavor. To get some intuition ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....a distortion rate function, which specifies the minimum possible distortion D( attainable when a source is encoded at a rate not exceeding bits per sample. An introduction to rate distortion theory can be found in [4] or [5, Appendix D] while a more in depth treatments are given in [6] and [7]. The most commonly used distortion measures are ,in#le letter distortion mea sures. A single letter distortion measure is an average of some nonnegative function of each source letter and the corresponding reproducing letter (rather than of the two ,equence, of letters) In the present study, ....

....each of the decision regions, i.e. D( Iq(X) Xl f(x)dx : Irk XlA(X)dX. Let Fk denote the probability that d z d. That is, define F : A(x)x. 1. 3) The entropy H of Q(z) which measures the minimum number of bits that must be used, on the average, to represent Q(z) without error [1][7], is given by H = F log 2 Fk. 1.4) 11 FK 1 dl FK 2 d2 d3 Q(x) dx 2 dx x Figure 1.3: Characteristic staircase function of a K level scalar quantizer. It is important to note that because the set of quantization regions is discrete, the en tropy can remain bounded even when the ....

Gallager, R.G., Information Theory and Reliable Communication. New York: Wiley, 1968.

....presented in [10] for the problem of synthesizing a probing feedback control associated with a filtering problem, some elementary results from information theory are discussed. A detailed account of information theory and its contribution to communication theory and applications are presented in [26] and [27] An essential notion of Shannon s information theory is entropy, including conditional entropy and mutual information. For random variables and given, the (differential) entropy of and the joint (differential) entropy of are defined as (1) where and are the (joint) density functions ....

....the mutual information of and . The following properties [which we will refer to as Property (P) can be easily proved. P1) P2) with equality iff independent. P3) for any measurable function . P) P4) if . iff independent. P5) for any constant . P6) if independent. References [26] and [27] provide a detailed account of these properties and their interpretation in information theory. A general framework for state estimation (filtering) in terms of the information theoretic measures defined in (1) 3) is provided next. For a feedback control given, the closedloop ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....give an upper bound for the performance applying powerful channel coding in the system of Fig. 1. We consider the capacity of the memoryless vector channel with discrete input and continuous complex valued output. The average mutual information , measured in bits per vector symbol, is obtained by [28] (6) where is the average pdf of the channel output. It is well known that for capacity calculation, an optimization over all free parameters has to be performed. On the one hand, the ring ratios , of the APSK constellation have to be chosen. On the other hand, the probabilities of the rings have ....

....rates of the component codes are properly chosen (see, e.g. 32] 15] and [33] Applying this procedure to the present vector fading channel, we arrive at a scheme, optimal in the sense of information theory. Since the mapping is bijective and using the mutual information chain rule [28], we can rewrite (7) as ( denotes the random variable corresponding to ) 8) with (9) Transmission of vectors of binary digits , over the physical channel is separated into the parallel transmission of individual digits over equivalent channels with capacity , provided that are known (see [14] ....

R. G. Gallager, Inform. Theory and Reliable Communication.New York: Wiley, 1968.

....finite state Markov channel (FSMC) model For Clarke s flat Rayleigh channel of (1) where the process 1# is stationary and ergodic, the definition of the capacity in 8 C. KOMNINAKIS AND R. D. WESEL: JOINT ITERATIVE CHANNEL ESTIMATION AND DECODING IN FLAT CORRELATED RAYLEIGH FADING Gallager [23] applies: 20) where and denote sequences of channel inputs and outputs respectively. Here we are interested in the constrained capacity for inputs from a finite uniform constellation, such as the 4 PSK we use. If the decoding delay is constrained to ....

R. G. Gallager. Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. Gallager, Information Theory and Reliable Communication.New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication,New York: Wiley, 1968.

No context found.

R. Gallager, Information Theory and Reliable Communication.New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Realiable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Realiable Communication. New York: Wiley, 1968.

.... Such bounds can generally be expressed in the form 2 E P(e) 22 E (2) where the quantity tc = log2 M represents the number of bits per message, E is the system reliability function, and and 2 are slowly varying functions of tc and other parameters such as Manuscript received January 25, 1973. This work was supported in part by the National Science Foundation and is based on portions of a dissertation submitted by S. Rosenberg to Columbia University in partial fulfillment of the requirements for the degree of Doctor of Engineering Science. S. Rosenberg is with Bell Laboratories, ....

....the receiver structure. Specifically we will examine the upper bound to the error probability P( 22 . 4) Using standard Chernoff bounding techniques [3 ] and assum ing equally likely equal energy orthogonal signals, the upper error bound may be written as P( M exp [po(t) pt) 5) Under the additional assumption of independent equal strength (flat) fading at all D detectors [a i = aj; i,j = 1, D] and using (1.18) from Part 1, the quantities o(t) and x( pt) are easily identified as [4] o(t) D In Nsn exp ( Ns) F(n,7,Ns,rr) t n=0 ( pt) D In Nsn exp ( Ns) ....

[Article contains additional citation context not shown here]

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968, pp. 137 and 529.

....its performance because the probability of channel errors occurring in the recovery area will be essentially statistically independent from that in the original burst for all tdcc m 1000. In Fig. 10, the performance of the new decoder is compared to that of the Gallager burst finding scheme [11] using essentially the same code. Although a dircct comparison is difficult because the Gallager scheme uses a less powerful decoder, we feel it is important to point out that the adaptive Viterbi decoder is not outperformed by the less complex system. One drawback of the Gallager scheme is its ....

R.G. Gallager, Infromation Theory and Reliable Communications. New York: Wiley, 1968.

....This is done in Fig. 10 for . In this figure, we have also plotted an extra point to represent the reconstruction error (zero) when all the subbands are present (i.e. If we connect the s in Fig. 10 using a smooth curve, the result will look very similar to a rate distortion curve [35] [36]. Like the rate distortion curve, the error versus sampling rate curve generated by the PCFB is a characteristic of the signal. The significance of the error versus sampling rate curve is that its shape shows how scalable a signal is. No matter how high is the order of the filterbank used, it is ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....t(x) pE(s) x) llg a lim sup log Z 2 L(w) n o lloga l log . Taking the lit as l approaches infity, we obtain (x) p( x) 0, 34) and since (34) holds for eve finite s, we have (35) for eve infinite sequence x. Using Huffman s coding scheme for input blocks of length l, it is easy to show [6] that l which when l tends to iinity becomes ( h( 36) for all x. Combining (35) with (36) completes the proof. Q.E.D. Proof of Theorem 4: Since x is drawn from an ergodic source, it follows that for eve positive integer l and for eve w A t where P(x,w) lim .ooP(x ,w) and Pr (w) is the ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....only property of the decoder we used is the restriction that the decoding regions not overlap. Thus the lemma holds independently of the decoder that one uses. A weaker form of the converse lemma can be found in [Mas] Speci cally he shows I(W ; B ) This combined with Fano s inequality, [Gal], gives us H( log M H(W ) I(W ; B 22 where H( is the entropy of a random coin with bias : This implies H( 1 Thus the rate R = log M C T is a necessary condition for the error to go to zero. We state the more general converse theorem now. Theorem ....

....P (F ) from a t=1 . By lemma 4.1 we know the channel t=1 and the code function distribution P (F ) uniquely de ne a channel from F T to B denoted by fP (B t j f t=1 . Thus we can directly apply Gallager s random coding error exponent theorem to the F T B channel. [Gal] 25 De nition 5.4 Given a channel fP (B t jf t=1 de ne the error exponent to be B 4 f T C A : Theorem 5.4 The average random coding error over (T ; e TR ) channel codes drawn according to P (F ) can be upperbounded as codebooks(T;e TR ) error) ....

[Article contains additional citation context not shown here]

R. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....is memoryless as indicated before and hence the achievable rate region assumes a single letter characterization. Note now that since the fading variables 5 No deletion of codebooks with average small normalized power as compared to SNR 0 ffl or large average power is done (in contrast to [99]) rather in those atypical events (for N 1) error is declared as in [95, ch. 10] SHAMAI AND WYNER: SYMMETRIC, CELLULAR, MULTIPLE ACCESS FADING CHANNELS PART I 1889 and the channel inputs are statistically independent, as the transmitters are not aware of the fading parameters and therefore ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

....p(x; y; z)p(x; u)p(y; u) p(u)p(x; y) x; y; u: p(x) 0;p(y) 0;p(u) 0 p(x; u)p(y; u) p(u) 1: The theorem is proved. ACKNOWLEDGMENT The authors wish to thank F. Matus for his careful reading of the manuscript. R. W. Yeung wishes to thank B. Hajek for his useful discussion. REFERENCES [1] T. S. Han, A uniqueness of Shannon s information distance and related non negativity problems, J. Comb. Inform. Syst. Sci. vol. 6, pp. 320 321, 1981. 2] F. Mat u s, Probabilistic conditional independence structures and matroid theory: Background, Int. J. General Syst. vol. 22, pp. ....

....at both the transmitter and receiver is achieved when the transmitter adapts its power, data rate, and coding scheme to the channel variation. The optimal power allocation is a water pouring in time, analogous to the water pouring used to achieve capacity on frequency selective fading channels [1], 2] We show that for independent and identically distributed (i.i.d. fading, using receiver side information only has a lower complexity and the same approximate capacity as optimally adapting to the channel, for the three fading distributions we examine. However, for correlated fading, not ....

[Article contains additional citation context not shown here]

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communications. New York: Wiley, 1968.

....residue code, and turbo codes of various lengths. II. BINARY SYMMETRIC CHANNEL Theorem 1 presents the sphere packing lower bound for the BSC, with crossover probability . Theorem 2 then provides an exact expression for the error for a random code operating on the BSC. Theorem 1 (Gallager, [12], p. 163) An code on the BSC has probability of error lower bounded by (1) where is defined as the maximum integer such that . For most and , there is no code providing equality in (1) The Hamming codes, Golay codes, and other perfect and quasiperfect codes are the only codes that achieve ....

R. Gallager, Information Theory and Reliable Communications. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communications. New York: Wiley, 1968.

....capacity region, colored Gaussian noise, intersymbol interference (ISI) I. INTRODUCTION T WO of the most celebrated results in Shannon theory are Gallager s capacity and corresponding water filling formula for single user Gaussian channels with intersymbol interference (ISI) and colored noise [1] and Cover s superposition coding technique for degraded broadcast channels [2] In this paper, we show that the capacity region of a Gaussian broadcast channel with ISI and colored noise is achieved using a strategy that combines superposition coding with an optimal power allocation achieved by ....

....graphically via a multilevel water filling, which assigns different water levels to the different channels depending on the relative priorities of the users. This multilevel water filling is reminiscent of Gallager s water filling strategy for a single user channel with ISI and colored noise [1]. We derive this optimal water filling strategy and the corresponding capacity region for several example channels. In practice, this optimal power allocation andcoding strategy couldbeimplemented using multicarrier modulation, which also uses a DFT to decompose an ISI channel ....

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager. Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information theory and reliable communication. New York : Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968, pp. 530--531.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information theory and reliable communication, New York: Wiley, 1968.

No context found.

R. Gallager, Information Theory and Reliable Communications.New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968, pp. 530--531.

No context found.

R. G. Gallager, Information Theory and Reliable Communication, New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Transmission. New York: Wiley, 1968.

No context found.

R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC