R. G. Gallager, "Variations on a theme by Huffman," IEEE Transactions on Information Theory, vol. 24, pp. 668--674, 1978.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... Binary prefix codes ending in a 1 , IEEE Trans. Inform. Theory, vol. 40, pp. 1296 1302, July 1994. 4] S. Chan and M. Golin, A dynamic programming algorithm for constructing optimal 1 ended binary prefix free codes, in Proc. IEEE Int. Symp. Information Theory, Boston, MA, 1998, p. 45. [5] D. A. Huffman, A method for the construction of minimum redundancy codes, Proc. IRE, vol. 40, pp. 1098 1101, 1952. 6] A. W. Marshall and I. Olkin, Inequalities: Theory of Majorization and Its Applications. New York: Academic, 1979. 7] R. T. Rockafellar, Convex Analysis. Princeton, NJ: ....

....2. We define R as the redundancy of a Huffman code. It is well known that 0 R 1. These bounds on R are the tightest possible when nothing is known about the source distribution. However, when partial knowledge about the source distribution is available, these bounds can be improved. Gallager [5] proved that if p1 , the probability of the most likely source symbol, is given, the upper bound on R can be improved. His result is summarized as p1 oe; if p1 2 0 p1 0H b (p1) if p1 (1) H b (x) 0x log 2 x 0 (1 0 x) log 2 (1 0 x) and oe =10 log 2 e log 2 (log 2 e) 0:086: This ....

R. G. Gallager, "Variations on a theme by Huffman," IEEE Trans. Inform. Theory, vol. IT-24, pp. 668--674, Nov. 1978.

....decode one symbol, update the frequency count of that symbol, update its table of Huffman codes and then decode the next symbol, and so on. Algorithms exist for efficiently updating the Huffman codes when small incremental changes to the probability estimates are made (as is the case here) 3] [4]. In spite of the widespread use of these algorithms in implementations, Adaptive Huffman coding is not renowned for its speed (nor for its compression performance) The overall structure of a word based Adaptive Huffman algorithm may take the form shown in Figure 1. The algorithm uses two tables ....

Gallager, R.G. "Variations on a Theme by Huffman." IEEE Trans. on Inf. Theory, IT24, 6 (Nov. 1978), pp. 668-674.

....Length of uncompressed message i q i k i q i k i i p i k i p i k i = f a Delta f b Delta ff opt Where: ffl f a is a function of the divergence between the samples. ffl f b is a function of the chosen algorithm and the p i . An upper bound of this function is given by [4]: 1 pmax 0:086 ffl ff opt is the lower bound of the compression rate: by the entropy theory [2] it is equal to . This formula separates the influence of the statistics ff opt and the influence of the algorithm f b on the compression efficiency. In our case, we suppose to know the ....

R. G. Gallager, "Variation on a Theme by Huffman," IEEE Transactions on Information Theory, vol 24, no 6, pp. 668-674, November 1978.

....) of the source: r = L Gamma H(p 1 ; p 2 ; p N ) p i n i p i log p i where log denotes the logarithm to base 2. It is well known that the redundancy of an optimal code satisfies 0 r 1. These bounds can be improved if one knows partial information on the source. Gallager [4], Johnsen [7] Capocelli et al. 3] Capocelli and De Santis [1] and Manstetten [10] considered the problem of upper bounding r when p 1 is known. Reza [9] and Horibe [5] considered the problem of upper bounding the redundancy when the least likely source letter probability p N is known. Their ....

....of labels on the path from root to leaf. The code tree is generated by the Huffman algorithm. Each internal node in the code tree has a probability defined as the sum of the probabilities of his two children. The external nodes have the probability of the corresponding source letter. Gallager [4] proved that an Huffman code tree has the sibling property, i.e. each node, except the root, has a sibling and all nodes can be listed in order of decreasing probability with each node being adjacent to its sibling. We number all the nodes, except the root, using the sibling property, so that for ....

[Article contains additional citation context not shown here]

R. G. Gallager, "Variation on a theme by Huffman", IEEE Trans. Inform. Theory, vol. IT-24, n. 6, pp. 668--674, Dec 1978.

....obtained with arithmetic coding without the time consuming arithmetic. It gives faster coding even than Huffman coding because of the especially simple prefix codes involved, and adaptive modeling is possible without the complicated data structure manipulations required in dynamic Huffman coding [2,3,9,15,16]. The main drawback to Golomb Rice coding is the limited class of distributions that can be modeled exactly, but even this is not a serious problem (unless one event s probability is close to 1) because the probabilities of the more probable events will be estimated fairly well. The idea of using ....

R. G. Gallager, "Variations on a Theme by Huffman," IEEE Trans. Inform. Theory IT--24 (Nov. 1978), 668--674.

....4 contains a preliminary performance analysis for a simple join query as commonly used in relational database systems. We offer our conclusions in Section 5. 1 2. Related Work A number of researchers have considered text compression schemes based on letter frequency, as pioneered by Huffman [10, 11, 16, 17]. Other recent research has considered schemes based on string matching [22, 30, 32, 33] Comprehensive surveys of compression methods and schemes are given in [2, 18, 27] Others have focussed on fast implementation of algorithms, parallel algorithms and VLSI implementations [12, 29] Few ....

R. G. Gallager, Variations on a Theme by Huffman, IEEE Trans. on Inf. Theory IT-24, 6 (1978), 668.

....decoder side that have an adapter algorithm to update or learn the coding parameters needed by coder algorithm . When the coder employs a Huffman code, the adapter coder is called an adaptive Huffman code. Faller [fal73] was the first to design an adaptive (one pass) Huffman code, and Gallager [gal78] publicised the code. Many, including Knuth and Vitter made subsequent suggestions. Adaptive Huffman codes dynamically maintain the binary tree employed in the design of static Huffman codes. As certain branches become more popular than others, the subtrees may be exchange and modified. A key ....

R. Gallager, "Variations on a theme by Huffman", IEEE Trans Inf. Theory, vol IT-24, no 6, November 1978, 668-674.

....with entropy H. Then the average codeword length E of the d ary, d 3, augmented code satisfies E H 1 1 d Gamma d d(d Gamma 1) 1 1 Gamma 1 e Delta ln d : 3) Before proving the Theorem, we report hereafter some known results that will be useful in the proof (see [7], 15] Property 1. If q a and q b are the probabilities of nodes a and b at the same level in a d ary Huffman code tree, and b is not a leaf, then dq a q b . Property 2. Let P be a probability distribution with entropy H. The redundancy r = E h Gamma H of a d ary Huffman code satisfies r ....

....Property a) of LS codes, cannot have 0 as codeword. Thus, the augmented code has E = E h = 1. Assume now n = d. The augmented code is f01; 1; d Gamma 1)g and its average codeword length E satisfies E Gamma H = 1 Gamma H(p 1 ; p 2 ; p d ) q; where q = minp i . Gallager [7] proved that H(x 1 ; x 2 ; x d ) dx (4) 12 where x = minx i and P x i = 1. Applying (4) we find that E Gamma H 1 Gamma q(d Gamma 1) and hence the theorem in the case n = d. Finally consider the case n d. Let ff be the length vector of C. Let q i , i = 0; 1; d Gamma ....

R. G. Gallager, "Variation on a theme by Huffman," IEEE Trans. Inform. Theory, vol. IT-24, n. 6, pp. 668--674, Nov. 1978.

....performance within the upper bound: if the quantizer index is i, then assign it a binary codeword with length d Gamma log P i e (the Kraft inequality ensures that this is always possible by simply choosing paths in a binary tree) Moreover, tighter bounds have been developed. For example Gallager [181] has shown that the entropy can be at most Pmax :0861 smaller than the average length of the Huffman code, when Pmax , the largest of the P i s, is less than 1 2. See [73] for discussion of this and other bounds. Since Pmax is ordinarily much smaller than 1 2, this shows that H(q(X) is ....

R. G. Gallager "Variations on a theme by Huffman," IEEE Trans. Inform. Theory, vol. 24, pp. 668-674, Nov. 1978.

....codes, but concentrated more on adaptive coding when discussing arithmetic coding. This was no accident. Clearly, arithmetic coding can also be used for static probability distributions; and conversely a variety of methods for dynamic Huffman coding have been described in the literature (Gallager, 1978; Cormack Horspool, 1984; Knuth, 1985; Vitter, 1989; Lu Gough, 1993) However, static arithmetic coding is not much faster than dynamic arithmetic coding, while dynamic Huffman coding is substantially slower than static (canonical) Huffman coding. Figure 3, derived from the results presented ....

.... minimum redundancy code can result in slightly better compression than an arithmetic code (Bookstein Klein, 1993; Moffat et al. 1994) A variety of authors have considered analytic bounds on the inefficiency of minimum redundancy codes, and shown them to be very accurate for most purposes (Gallager, 1978; Capocelli et al. 1986; Capocelli De Santos, 1991; Manstetten, 1992) 2.8 Binary Alphabet Coding One important domain in which arithmetic coding is essential is binary alphabet coding, and methods for this problem are worthy of special mention. When the input alphabet is binary pixels in a ....

Gallager, R.G. (1978). Variations on a theme by Huffman. IEEE Transactions on Information Theory, IT-24(6):668--674.

....are all conditional on the context s in which the character a appears. It is well known that Huffman encoding of an independent source gives close to the optimal entropy rate for the source. For example we have the following lemma quoted in Welsh [9] and originally proved by Gallager [4]. Preliminary Approach 4 Lemma 3.1 Let r be the difference between the expected length of a Huffman code per input symbol minus the entropy for an independent source with probabilities p 1 ; p n . Then r p max log[2(log e) e] p max 0:086; where p max = max i fp i g. We now ....

R. G. Gallager, Variations on a theme by Huffman, IEEE Trans. Info. Theory, IT--24 (1978) 668--674.

....Alphabets and Binary Alphabets. 2.1 Large Alphabet A database is often composed from an alphabet of a moderate to large number of coding units, for example a natural language alphabet, numbers, and punctuation for a textual database. In this case, a bound by Gallager is often applicable. Gallager [19] showed that the redundancy of a Huffman code is at most p 1 0:086, where p 1 is the probability of the most frequent codeword. The redundancy is defined as the difference between the average length of a Huffman code and the corresponding entropy, or for our application, by how much Huffman ....

.... 4: Comparison of processing speed Note that encoding for arithmetic codes took more than twice as long as for Huffman codes, and decoding up to 10 times as long Since arithmetic codes can easily be used with an adaptive model, it is perhaps more fair to compare them with adaptive Huffman codes [19], 42] 30] as done in [43] Our results (columns headed adap H ) were however different from those reported in [43] yielding a decoding speed up to 6 times faster for adaptive Huffman codes than for arithmetic codes. There have been attempts to improve the speed of arithmetic codes, either ....

Gallager R.G., Variations on a theme by Huffman, IEEE Trans. on Inf. Th., IT--24 (1978) 668--674.

....r = L Gamma H(p 1 ; p 2 ; p N ) N X i=1 p i n i N X i=1 p i log p i where log denotes the logarithm to base 2. It is well known that the redundancy of an optimal code satisfies 0 r 1. These bounds can be improved if one knows partial information on the source. Gallager [5], Johnsen [8] Capocelli et al. 3] Capocelli and De Santis [1] and Manstetten [11] considered the problem of upper bounding r when the most likely source letter probability p 1 is known. Reza [10] and Horibe [6] considered the problem of upper bounding the redundancy when the least likely ....

....node in the code tree has a probability defined as the sum of the probabilities of his two children. The external nodes have the probability of the corresponding source letters. The Huffman encoding algorithm provides an optimal code for a given source, by constructing its code tree. Gallager [5] proved that an Huffman code tree has the sibling property, i.e. each node, except the root, has a sibling and all nodes can be listed in order of decreasing probability with each node being adjacent to its sibling. We number all the nodes, except the root, using the sibling property, so that for ....

[Article contains additional citation context not shown here]

R. G. Gallager, "Variation on a theme by Huffman", IEEE Trans. Inform. Theory, vol. IT-24, n. 6 (1978), pp. 668--674.

....for the source. It is well known that the redundancy of any source satisfies 0 r 1. These bounds can be improved if one knows partial information on the source. The problem of finding better bounds when some of the probabilities in the probability list are known, first considered by Gallager [6], has been extensively studied in several papers, such as [1, 2, 3, 4, 5, 6, 9, 11, 12, 13, 16] Much of the work done has been concentrated on binary codes. Some of the results obtained for binary codes have been generalized to D ary codes. A lower bound is said to be achievable if there exists ....

....source satisfies 0 r 1. These bounds can be improved if one knows partial information on the source. The problem of finding better bounds when some of the probabilities in the probability list are known, first considered by Gallager [6] has been extensively studied in several papers, such as [1, 2, 3, 4, 5, 6, 9, 11, 12, 13, 16]. Much of the work done has been concentrated on binary codes. Some of the results obtained for binary codes have been generalized to D ary codes. A lower bound is said to be achievable if there exists a source whose redundancy achieves the bound. A lower bound is said to be tight if for any ....

[Article contains additional citation context not shown here]

R. G. Gallager, "Variation on a theme by Huffman", IEEE Trans. Inform. Theory, vol. IT-24, n. 6 (1978), pp. 668--674.

No context found.

R. G. Gallager, "Variations on a theme by Huffman," IEEE Transactions on Information Theory, vol. 24, pp. 668--674, 1978.

No context found.

R. S. Gallager, "Variations on a theme by Huffman," IEEE Trans. Inform. Theory, vol. IT-24, pp. 668--674, Nov. 1978.

No context found.

R. G. Gallager, "Variations on a theme by Huffman," IEEE Trans. Inform. Theory, vol. 24, pp. 668--674, November 1978.

No context found.

R.G. Gallager, `Variations on a theme by Huffman', IEEE Transactions on Information Theory, IT-24, (6), 668--674, (November 1978).

No context found.

R.G. Gallager,Variations on a theme by Huffman, in: IEEE Trans. Inf. Theory, 24, 6 (Nov.), 1978, 668-674.

No context found.

R. G. Gallager, "Variations on a theme by Huffman," I.E.E.E. Trans. Inf. Thy., IT-24, 668-674, 1978.

No context found.

R. G. Gallager, "Variations on a theme by Huffman," IEEE Trans. Inform. Theory, Vol. IT-24, pp. 668-674, Nov. 1978.

No context found.

R. G. Gallager, "Variations on a theme by Huffman," I.E.E.E. Trans. Inf. Thy., IT-24, 668-674, 1978.

No context found.

R. S. Gallager, "Variations on a theme by Huffman," IEEE Trans. on Info. Th., vol. IT-24, pp. 668 674, Nov. 1978.

No context found.

R. G. Gallager, `Variations on a theme by Huffman', IEEE Transactions on Information Theory, IT-24, (6), 668--674 (1978).

No context found.

Gallager, R.G. (1978) Variation on a theme by Huffman. IEEE Trans. Inform. Theory, 24 (6), 668--674.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC