I. Csisz ar and J. K orner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....by T the set of all n tuples x that are typical according to PX , i.e. T X = N(xjx) PX (x) 8 x 2 X ; and N(xjx) 0 whenever PX (x) 0g where N(xjx) is the number of occurrences of the letter x in the n tuple x. In the sequel, we will use the following well known results [2]. For any x 2 T X and any exp[ n(I(X; U) u ) u: x;u)2T 0 PU (u) exp[ n(I(X; U) 131) 32 where = and u = u ( both vanish as , 0. Similarly, for any pair (x; u) 2 T and any exp[ n(I(X; V jU) vju ) v: x;u;v)2T 00 P V jU ....

I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Academic Press, London, 1981.

....cg are both upper bounded by a function (c) for all n n c , where (c) and n c do not depend on either or , and (c) tends to zero as c 1. For the parameter , consider the estimator = 1 N n =n, calculated from the n observations of the Bernoulli process y 1 ; y n . Using the fact [24] that for ; 2 [0; 1] DB ( jj ) log (1 ) log 1 1 2( ln 2 ; 17) the Cherno bounding technique gives nj j cg expfn ln 2 min DB ( jj )g expf2n min ( g = exp( 2c ) 18) As for the parameter , consider the estimator ....

I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.

....throughout the sequel, log( log 2 ( For a given positive integer k, let U = U 1 ; U k ) U i 2 B = f0; 1g, i = 1; k, denote a string of k random bits, drawn from the binary symmetric source, independently of X . Since we will rely quite heavily on the method of types [1] in this paper, we next describe the notation that will be used in this context: For a given source vector x 2 A the empirical probability mass function (EPMF) is the vector Qx = fq x (a) a 2 Ag, where q x (a) is the relative frequency of the letter a 2 A in the vector x. The type class Tx of ....

.... in for , and so, the maximum de ning F is attained for = 0 = 1 ) Thus, the numerator of the expression at hand is upper bounded by exp 2 fjT Q j [ h( 0 ) 1 )h( 0 ) g ; where 0 = 0 ) 1 ) The denominator, on the other hand, is lower bounded [1] by: jT Q j 1 exp 2 fjT Q j h( g: 29) When plugging the upper bound on the numerator and the lower bound on the denominator into eq. 27) the exponent of the denominator is subtracted from that of the numerator and we obtain: h( 0 ) 1 )h( 0 ) h( D( 0 k ) 1 )D( ....

[Article contains additional citation context not shown here]

I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.

..... In this case, N(P;m) m jAj 1) jAj 1) m ] The type classes in this special case will be referred to as elementary type classes. Notice that for any family P, the type classes are given by unions of elementary type classes. As a result, we will rely quite heavily on the method of types [8]. Similar notations will be used for types of sequences y , with m, x, and X being replaced by n, y and Y , respectively. Next, for every type class T 2 T , we de ne P (T ) P ( x ) jT j P (x ) 1) where x is a sequence in T . Given some enumeration of T , let ....

....exponentially fast with n, and therefore the lower bound of Theorem 1 behaves like n(H R) within an exponentially vanishing term. To show that the probability of the set D behaves as claimed, it suces to prove the following lemma, which generalizes analogous results for elementary type classes [8]. Lemma A.1 For any 0, let D n j nHj ng. Then, log PrfD g 0 : Proof. By the de nition of a class type, 0 log jT x n j log = log P (T x n) A.8) and PrfP (T X n ) 2 g N(P;n)2 31 where N(P;n) grows polynomially fast with n (as the number of type classes is no ....

[Article contains additional citation context not shown here]

I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981. 36

....result. will denote the set of all probability distributions on the set , and will denote the set of all types, i.e. iff is an integer for all . will denote the set of all DMC s with input alphabet and output alphabet . The notation for all standard information theoretic quantities is that of [6]. All logarithms and exponentials will be to the base two. Throughout, will denote the set of integers . B. Definition of a Protocol We will now formulate precisely the problem of generating common randomness over noisy channels in the absence of external sources. Later, in Section II D, we ....

I. Csisz ar and J. K orner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.

....(i) log e 2)R(D) sup 0 [ D ( where ( sup Q P;Q ( ii) Let be chosen as in (10) If R( is di erentiable at D, then (D) 3.2 Proofs in the Discrete Case For the proof of Theorem 1 we will need the following lemma. It easily follows from Theorem 3. 7 in Chapter 2 of [6] (see the Appendix) Recall the notation P i = P (a i ) and ij = a i ; a j ) Lemma 3. A probability mass function Q on A achieves the in mum in (9) if and only if there exists a 0 such that the following all hold: a) D: b) If we de ne, for a i ; a j 2 A, W (a i ; ....

I. Csiszar and J. Korner. Information Theory: Coding Theorems for Discrete Memoryless Systems. Academic Press, New York, 1981.

....1. In section 5 we discuss results of a similar nature for the erasure channel. 5 3 From theorem 1 to theorem 3 Let us rst gather here a few properties on and which are very useful for proving several facts and propositions of this paper (for a proof of these statements see for instance [5] lemma 5.2 p.88) Lemma 1 i. is a positive and concave function on (0; 1) and (x) 1 x) for every x 2 (0; 1) ii. iii. 1 on (0; 1) iv. lim s 0 (s) s) 1, v. lim s 1 s (s) s(1 (s) 1. These properties can be used to derive theorem 2 from ....

I. Csiszar, J. Korner, Information Theory Coding theorems for discrete memoryless systems, Academic press, 1981.

....(log e 2)R(D) sup 0 [ D ( where ( sup Q P;Q ( ii) Let be chosen as in (10) If R( is di erentiable at D, then (D) 8 3.2 Proofs in the Discrete Case For the proof of Theorem 1 we will need the following lemma. It easily follows from Theorem 3. 7 in Chapter 2 of [6] (see the Appendix) Recall the notation P i = P (a i ) and ij = a i ; a j ) Lemma 3. A probability mass function Q on A achieves the in mum in (9) if and only if there exists a 0 such that the following all hold: a) D: b) If we de ne, for a i ; a j 2 A, W (a i ; ....

I. Csiszar and J. Korner. Information Theory: Coding Theorems for Discrete Memoryless Systems. Academic Press, New York, 1981. 14

....with denominator n, will be denoted by Q n . The type class T x of a vector x is the set of all vectors x 0 2 X such that Q x 0 = Q x . When we need to attribute a type class to a certain rational PMF Q 2 Q n rather than to a sequence in X , we shall use the notation TQ . It is well known [1] that the number of type classes of n vectors is bounded by (n 1) where jX j denotes the cardinality of X . The standard reference about the method of types is the book by Csisz ar and K orner [1] Finally, throughout the sequel, o(n) designates a quantity that grows sub linearly with n, ....

....PMF Q 2 Q n rather than to a sequence in X , we shall use the notation TQ . It is well known [1] that the number of type classes of n vectors is bounded by (n 1) where jX j denotes the cardinality of X . The standard reference about the method of types is the book by Csisz ar and K orner [1]. Finally, throughout the sequel, o(n) designates a quantity that grows sub linearly with n, i.e. o(n) n 0 as n 1. 3 Main Results For a given cipher system , let P (yjx) denote the induced conditional probability of the cryptogram y given the plaintext x. Similarly, let P (x; y) P (x)P ....

I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.

....p with an optimal code for the model distribution q. The symmetric divergence, i.e. D (p; q) DKL [pjjq] DKL [qjjp] suffers from similar sensitivity problems and lacks a clear statistical meaning. 2. 1 The two sample problem Direct Bayesian arguments, or alternately the method of types [CK81], suggest that the probability that there exists one source distribution M for two independently drawn samples, x and y, Leh59] is proportional to Z d (M ) Pr (xjM ) Delta Pr (yjM ) Z d (M ) Delta 2 Gamma(jxjD KL [p x jjM] jyjDKL [py jjM ] 2) where d(M ) is a prior density of all ....

I. Csisz'ar and J. Krorner. Information Theory: Coding Theorems for Discrete Memoryless Systems, Academic Press, New-York 1981.

....the match appears the earliest. To describe X n 1 to the decoder with distortion D or less we then describe two things: a) the index of the codebook in which the earliest match was found, and (b) the position i n of this earliest match. Since there are at most polynomially many n types (cf. [25][24] the rate of the description of (a) is asymptotically negligible. Moreover, since the set of n types is asymptotically dense among probability measures on A, we eventually do as well as if we were using the optimum codebook distribution Q n . Theorem 11. Pointwise Universal Coding ....

I. Csiszar and J. Korner. Information Theory: Coding Theorems for Discrete Memoryless Systems. Academic Press, New York, 1981.

....the capacity of a fading channel with arbitrary variation is at most the capacity of a time invariant channel under the worst case fading conditions. More details about the capacity of time varying channels under these assumptions can be found in the literature on Arbitrarily Varying Channels [7], 8] The remainder of this correspondence is organized as follows. The next section describes the system model. The capacity of the fading channel under the different side information conditions is obtained in Section III. Numerical calculation of these capacities in Rayleigh, log normal, and ....

I. Csisz ar and J. K orner, Information Theory: Coding Theorems for Discrete Memoryless Channels. New York: Academic, 1981.

.... channel and more generally a finite state Markov channel with known channel statistics [20, 21] and the capacity of decorrelating channels [3, 22, 16, 23] Capacity of the Arbitrarily Varying Channel, where the channel varies arbitrarily over a set of memoryless channel states, was obtained in [24, 25] and the references therein. Lack of side information at the receiver prevents the decoder from using this information in its decoding strategy, and lack of side information at the transmitter prevents adaptation to the changing channel. Thus channel capacity without side information provides a ....

I. Csisz'ar and J. K'orner, Information Theory: Coding Theorems for Discrete Memoryless Channels. New York: Academic Press, 1981.

....the properties of the function that we need. Lemma 1 The function = 1 satis es the following properties. i. is a positive and concave function on (0; 1) and (x) 1 x) for every x 2 (0; 1) ii. 0 = 1 , iii. 00 = 1 on (0; 1) for a proof of Lemma 1 see for instance [4] Lemma 5.2 p.88. Let u = and recall that c is such that c = 1=2, by de nition when c 2 R and by (3) when c = 1) By integrating (6) between and c (where c ) we have Z 1=2 ( du (u) Z c D 2t 2 dt: By applying property iii. of Lemma 1 we obtain Z ....

I. Csiszar and J. Korner, Information Theory Coding theorems for discrete memoryless systems, Academic press, 1981.

No context found.

I. Csisz ar and J. K orner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.

No context found.

I. Csisz ar and J. K orner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Akademiai Kiado, Budapest, 1981.

No context found.

I. Csisz ar and J. Krner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.

No context found.

I. Csisz ar and J. K orner, Information Theory: Coding Theorems for Discrete Memoryless Systems. Budapest: Akad emiai Kiad o, 1981.

No context found.

I. Csiszar and J. Korner, Information Theory : coding theorems for discrete memoryless systems, Akademiai Kiado, Budapest, 1981.

No context found.

I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Akademiai Kiado, Budapest, 1981.

No context found.

I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Academic Press 1981.

No context found.

I. Csisz,lr and J. KiSrner, Information Theory: Coding Theorem for Discrete Memoryless Systems. Budapest, Hungary: Akadmiai Kiad6, 1981.

No context found.

I. Csiszar and J. Korner, Information theory: coding theorems for discrete memoryless systems, Academic Press, 1981.

No context found.

I. C. ar and J. anos Korner, Information Theory: Coding Theorems For Discrete Memoryless Systems, New York, NY: Academic Press, 1981.

No context found.

I. Csiszar and J. Korner. Information Theory: Coding Theorems for Discrete Memoryless Systems. Academic Press, New York, 1981.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC