D. E. Knuth. Dynamic Huffman coding. In Journal of Algorithms,6:163180, June 1985.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....the parameters. For example, in the tree problem, R = n, A = 1, and D = n will suffice as upper bounds. In the online path problem, A = m and D = 2n will suffice, assuming times are in [0; 1] and of course we only need to consider paths of length at most n. The Adaptive Huffman coding problem [12] is not normally considered as an online algorithm. But it fits naturally into the framework. There, one wants to choose a prefix tree for each symbol in a message, on the fly without knowledge of the sequence of symbols in advance. The cost is the length of the encoding of the symbol, i.e. ....

D. Knuth. Dynamic Huffman Coding. J. Algorithms, 2:163-180, 1985.

....internal nodes and leaf nodes since leaf blocks do not need a rtChild value. If storage is dynamically allocated, as opposed to preallocated via arrays, it is typically much less. 3. PASCAL CODE The basic implementation of Algorithm A is along the lines of the implementation of Algorithm FGK in [2]. The primary difference between the two is that Algorithm A uses the implicit node numbering and the floating tree data structure in order to maintain the invariant defined in Section 1. ACM Transactions on Mathematical Software, Vol. 15, No. 2, June 1989. The basic loop for encoding and ....

....one for internal nodes. One nice feature of a floating tree, due to the use of implicit numbering, is that the parent of nodes 2j I and 2j is less than the parent of nodes 2j I and 2j 2 in both the implicit and explicit numberings. Such an invariant is not maintained by the data structure in [2], for example. ACM Transactions on Mathematical Software, Vol. 15, No. 2, June 1989. ....

KNUTH, D.E. Dynamic Huffman coding. J. Algorithms 6 (1985), 163-180.

....obtained with arithmetic coding without the time consuming arithmetic. It gives faster coding even than Huffman coding because of the especially simple prefix codes involved, and adaptive modeling is possible without the complicated data structure manipulations required in dynamic Huffman coding [2,3,9,15,16]. The main drawback to Golomb Rice coding is the limited class of distributions that can be modeled exactly, but even this is not a serious problem (unless one event s probability is close to 1) because the probabilities of the more probable events will be estimated fairly well. The idea of using ....

D. E. Knuth, "Dynamic Huffman Coding," J. Algorithms 6 (June 1985), 163--180.

....4 contains a preliminary performance analysis for a simple join query as commonly used in relational database systems. We offer our conclusions in Section 5. 1 2. Related Work A number of researchers have considered text compression schemes based on letter frequency, as pioneered by Huffman [10, 11, 16, 17]. Other recent research has considered schemes based on string matching [22, 30, 32, 33] Comprehensive surveys of compression methods and schemes are given in [2, 18, 27] Others have focussed on fast implementation of algorithms, parallel algorithms and VLSI implementations [12, 29] Few ....

D. E. Knuth, Dynamic Huffman coding, J. of Algorithms 6(1985), 163.

....since Huffman s paper. Hamming [1] discusses the information theory behind some of the basic compression techniques. Storer s book [7] gives a nice survey of the field. Adaptive compression schemes that adaptively adjust to changing source symbol statistics have been developed; Knuth s paper [6] gives one example. Jones [3] presents another interesting method, based on the use of splay trees, and provides an excellent discussion of the ways in which data compression algorithms may be used as an encryption techniques. The fundamental question we address is: How hard is it to ....

Donald E. Knuth. Dynamic Huffman coding. J. Algorithms, 6(2):163--180, February 1985.

....appended) and corresponding zero free output. For clarity, COBS code bytes are shown shaded. of as a general purpose compression algorithm, and it is not intended to compete with more sophisticated (and more computationally expensive) compression algorithms such as Huffman encoding [13] [20], and Lempel Ziv [21] 37] Although, like PPP, these compression algorithms may have good average performance, for some data they can make the packet bigger instead of smaller [14] and it can be hard to predict how much bigger they may make a packet in the worst case, which is contrary to our ....

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, vol. 6, pp. 163--180, 1985.

....sample, without recourse to the storage of code tables, as would be the case with unstructured, generic Huffman codes. In an adaptive mode, a structured family of codes further relaxes the need of dynamically updating code tables due to possible variations in the estimated parameters (see, e.g. [37]) 3.3.1 Golomb codes and optimal prefix codes for the TSGD Golomb codes were first described in [29] as a means for encoding run lengths. Given a positive integer parameter m, the mth order Golomb code Gm encodes an integer y 0 in two parts: a unary representation of by=mc, and a modified ....

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, vol. 6, pp. 163--180, 1985.

....that the height of the tree cannot exceed L. Throughout this work, we use optimal code tree with restricted maximal height L to denote a tree that solves the L Gammarestricted prefix code problem. 2. 2 A Characterization for the Huffman Trees Based on Gallager s sibling property [Gal78] Knuth [Knu85], has shown that a binary tree T with n leaves is a Huffman tree, if and only if it satisfies the following properties: i) To the n leaves have been assigned the probabilities p 1 ; p n is some order, and to each internal node has been assigned a probability equal to the sum of the ....

Knuth,D.E.Dynamic Huffman Coding, Journal of Algorithms 6,2 (1985), 163-180.

....determines the mapping from symbol alphabets to codewords based upon a running estimate of the alphabet symbol weights. The code is adaptive, just changing to remain optimal for the current estimates. Adaptive Huffman coding was first conceived independently by Faller and Gallager [1, 2] Knuth [3] proposed an efficient implementation to generate such codes. The resulting algorithm is referred as the FGK algorithm. Another version for the adaptive Huffman coding is the algorithm described by Vitter [12, 13] Both algorithms require O(l) time for each encoding and decoding operation, where l ....

....for the FGK algorithm. We also present an example where D t = S t 2t Gamma 2k Gamma 3b(t Gamma k) kc Gamma dlog(k 1)e, what shows that the proposed bound is asymptotically tight. These results explain the good performance of FGK that some authors have observed through practical experiments [3, 12]. The paper is organized as follows. In section 2, we present basic concepts of the FGK algorithm. In section 3, we show our improved analysis of the algorithm. In section 4, we prove that our bound is asymptotically tight. Finally, in section 5, we comment on our findings. 2 The FGK Algorithm ....

Knuth, D.E., Dynamic Huffman Coding, Journal of Algorithms 6,2(1985), 163180.

....coding when discussing arithmetic coding. This was no accident. Clearly, arithmetic coding can also be used for static probability distributions; and conversely a variety of methods for dynamic Huffman coding have been described in the literature (Gallager, 1978; Cormack Horspool, 1984; Knuth, 1985; Vitter, 1989; Lu Gough, 1993) However, static arithmetic coding is not much faster than dynamic arithmetic coding, while dynamic Huffman coding is substantially slower than static (canonical) Huffman coding. Figure 3, derived from the results presented by Moffat et al. 1994) shows the ....

Knuth, D.E. (1985). Dynamic Huffman coding. Journal of Algorithms, 6:163--180.

....sample, without recourse to the storage of code tables, as would be the case with unstructured, generic Huffman codes. In an adaptive mode, a structured family of codes further relaxes the need of dynamically updating code tables due to possible variations in the estimated parameters (see, e.g. [41]) Golomb codes were first described in [32] as a means for encoding run lengths. Given a positive integer parameter m, the mth order Golomb code Gm encodes an integer y 0 in two parts: a unary representation of by=mc, and a modified binary representation of y modm (using blog mc bits if y 2 ....

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, vol. 6, pp. 163--180, 1985.

....experiments the Centroid method was tested in combination with three well known compression techniques: Huffman encoding, Arithmetic coding, and Lempel Ziv compression. The first two are entropy based techniques; the third is a dictionary based compression scheme. Huffman encoding (Huffman, 1952; Knuth, 1985) is one of the oldest and most widely used compression methods. Arithmetic coding (Langdon, 1984; Witten et al. 1987; Moffat et al. 1995) is slower and more difficult to implement; however, it outperforms the Huffman method in compression ratios. Lempel Ziv compression (Ziv and Lempel, 1977, ....

Knuth, D.E., 1985. Dynamic Huffman coding. J. Algorithms 6, 163-180.

....of every given source symbol, without recourse to the storage of code tables for large alphabets. This property makes the family attractive for use in adaptive schemes [13, 14, 3] since it avoids the need to dynamically update code tables as in traditional adaptive Huffman coding (see, e.g. [15]) Thus, the economy of parameters of the TSGD is reflected in the simplicity of the codes, and only a small number of variables need to be updated, and simple rules applied, to adaptively select a code for each sample. The optimal family of prefix codes derived here enables the adaptive ....

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, vol. 6, pp. 163--180, 1985.

....into one composite symbol (this is known as run length encoding) Often, symbol probabilities may not be stationary (for example, in an image, they may change from area to area) In this case, Huffman coding, which assumes a stationary source, is suboptimal. Dynamic Huffman coding schemes [Knu85, Vit87] adaptively adjust the codewords through the encoding process, to take into account varying symbol probabilities, but their implementations are complex. 35 Practical implementations of Huffman coding usually make several simplifications (such as limiting the maximum codeword length) for ....

Knuth, D. E. Dynamic Huffman coding. J. Algorithms, 6:163--180, 1985.

....advantage is almost lost with such a bound, so one will rarely prefer this alternative to the simple and almost as efficient fixed length code. We have therefore decided to test the compression efficiency of the new method empirically on various real life weight distributions, similarly to Knuth [17], who checked his dynamic Huffman coding algorithm on, e.g. a file of Grimm s Fairy Tales. For any given set of n weights, the Huffman tree was built, with depth K. Using then Garey s algorithm, the optimal B restricted trees were constructed for all possible values of B, dlog 2 ne B K Gamma ....

Knuth D.E., Dynamic Huffman coding, J. of Algorithms 6 (1985) 163--180.

.... of processing speed Note that encoding for arithmetic codes took more than twice as long as for Huffman codes, and decoding up to 10 times as long Since arithmetic codes can easily be used with an adaptive model, it is perhaps more fair to compare them with adaptive Huffman codes [19] 42] [30], as done in [43] Our results (columns headed adap H ) were however different from those reported in [43] yielding a decoding speed up to 6 times faster for adaptive Huffman codes than for arithmetic codes. There have been attempts to improve the speed of arithmetic codes, either approximating ....

Knuth D.E., Dynamic Huffman coding, J. of Algorithms 6 (1985) 163--180.

....Rajagopalan, Venkatesan, and Wei the codewords while minimizing the average codeword length (under the alphabetic constraint) For a source with unknown statistics, one needs to use an adaptive or dynamic algorithm. If there is no alphabetic order, adaptive versions of Huffman s algorithm [4] 5][13][17] can be used. However, there are relatively fewer works on adaptive alphabetic coding trees. We describe several applications of the splay tree idea to this problem and analyze their compression efficiency in information theoretic terms. For example, the average codeword length of the ....

D. E. Knuth, Dynamic Huffman coding, J. Algorithms, 6 (1985) 163--180.

....especially when used with adaptive models [5] A single bit error in the encoded file causes the decoder s internal state to be in error, making the remainder of the decoded file wrong. In fact this is a drawback of all adaptive codes, including Ziv Lempel codes and adaptive Huffman codes [12,15,18,26,55,56]. In practice, the poor error resistance of adaptive coding is unimportant, since we can simply apply appropriate error correction coding to the encoded file. More complicated solutions appear in [5,20] in which errors are made easy to detect, and upon detection of an error, bits are changed ....

....the coding trees efficiently without using excessive space. The smallest average number of events coded per input symbol occurs when the tree is a Huffman tree, since such trees have minimum average weighted path length; however, maintaining such trees dynamically is complicated and slow [12,26,55,56]. In Section 3.3 we present a new data structure, the compressed tree, suitable for binary encoding of multi symbol alphabets. 2.3 Modeling for text compression 9 2.3 Modeling for text compression Arithmetic coding allows us to compress a file as well as possible for a given model of the ....

[Article contains additional citation context not shown here]

D. E. Knuth, "Dynamic Huffman Coding," J. Algorithms 6 (June 1985), 163--180.

....internal nodes. One nice feature of a floating tree, due to the use of implicit numbering, is that the parent of nodes 2j Gamma 1 and 2j is less than the parent of nodes 2j 1 and 2j 2 in both the implicit and explicit numberings. Such an invariant is not maintained by the data structure in [Knuth, 1985], for example. ....

D. E. Knuth. "Dynamic Huffman Coding," Journal of Algorithms, 6 (1985), 163--180.

....: 186 6.5 Conclusions and future work : 190 6. 1 Introduction The most successful methods for lossless compression of data, such as arithmetic coding [55, 116, 80] Lempel Ziv coding [120] or dynamic Huffman coding [35, 54, 111], are all adaptive (see [6] for an extensive review of lossless compression) While the initial work on entropy coding (e.g. Huffman coding) relied on knowing, or measuring, the source distribution, adaptive schemes make no prior assumptions on the source statistics, which the coders try to learn. ....

.... of noisy behavior if the source is stationary [83] As another example of adaptation in the context of lossless coding, it has been shown that the Huffman coding tree can be modified on the fly so that the code would adapt to changing statistics, or learn them starting with no prior knowledge [35, 54, 111]. A first approach to generate these statistics would be to choose the number of samples N over which symbol occurrences are counted. However, a fully adaptive scheme would also require a procedure to change N , if necessary, during the coding process in order to improve the performance (we would ....

D. E. Knuth. Dynamic Huffman coding. J. Algorithms, (6):163--180, 1985.

.... or semi adaptive model, Huffman coding, ShannonFano coding and arithmetic coding attempt to assign short codes to frequently occurring input strings [F49, H52, FW78] Dynamic Huffman coding and arithmetic coding are examples of statistical coders that work in conjunction with an adaptive model [K82, V87, RL79, L84, WNC87]. Codeword based statistical coders replace input strings by codewords to obtain a more compact representation of the input. Huffman coding and ShannonFano coding are codeword based. However, in some compression schemes, such as arithmetic coding, it is not possible to identify the particular ....

....with the systolic array. Huffman coding provides better compression (19 to 38 ) but operates at a limited data rate. Dynamic Huffman coding and arithmetic coding yield far better compression but even the most sophisticated implementations operate at data rates below 15 KBytes per second [MP88, K82, V87]. The systolic decoder for move to front list compression cannot simply mirror the encoder since it is impossible to update the dictionary until after the input has passed PE 1 (except if the input is 1 ) and has been decoded at some processor later in the array. By using a two way communication ....

[Article contains additional citation context not shown here]

Knuth, D. E. Dynamic Huffman coding. J. Algorithms 6 (1982), 163--180.

....obtained with arithmetic coding without the time consuming arithmetic. It gives faster coding even than Huffman coding because of the especially simple prefix codes involved, and adaptive modeling is possible without the complicated data structure manipulations required in dynamic Huffman coding [2,3,9,15,16]. The main drawback to Golomb Rice coding is the limited class of distributions that can be modeled exactly, but even this is not a serious problem (unless one event s probability is close to 1) because the probabilities of the more probable events will be estimated fairly well. The idea of using ....

D. E. Knuth, "Dynamic Huffman Coding," J. Algorithms 6 (June 1985), 163--180.

....of EXAMPLE. described in Section 4. The performance of this implementation is discussed in Section 6. 4. ADAPTIVE HUFFMAN CODING Adaptive Huffman coding was first conceived independently by Faller and Gallager [Faller 1973; Gallager 1978] Knuth contributed improvements to the original algorithm [Knuth 1985] and the resulting algorithm is referred to as algorithm FGK. A more recent version of adaptive Huffman coding is described by Vitter [Vitter 1987] All of these methods are defined word schemes which determine the mapping from source messages to codewords based upon a running estimate of the ....

....a proof that the time required for 8 5 3 2 1 0 1 1 21 13 8 5 3 2 1 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 a e space d b f c Figure 4.3 Tree formed by algorithm FGK for ensemble e eae de eabe eae dcf . each encoding or decoding operation is O(l) where l is the current length of the codeword [Knuth 1985]. It should be noted that since the mapping is defined dynamically, during transmission, the encoding and decoding algorithms stand alone; there is no additional algorithm to determine the mapping as in static methods. 4.2 Algorithm V The adaptive Huffman algorithm of Vitter (algorithm V) ....

[Article contains additional citation context not shown here]

Knuth, D. E. 1985. Dynamic Huffman Coding. J. Algorithms 6, 2 (June), 163--180.

....34.16 2.13 63.60 Factor 47.43 31.86 0.09 73.74 Figure 10.26: Sizes of texts compressed with three algorithms. 10.8. DEFINING TERMS 31 The dynamic version has been discovered independently by Faller (1973) and Gallager (1978) Practical versions are given by Cormack and Horspool (1984) and Knuth (1985). A precise analysis leading to an improvement is presented in (Vitter, 1987) The command compact of UNIX implements the dynamic Huffman coding. It is unclear to whom precisely should be attributed the idea of data compression using arithmetic coding. It is sometimes refer to Elias (1963) and ....

Knuth, D.E. 1985. Dynamic Huffman coding. J. Algorithms. 6:163--180.

....In practice there are several ways to do this: ffl Periodically restarting the model. This often discards too much information to be effective, although Cormack and Horspool find that it gives good results when growing large dynamic Markov models [8] ffl Using a sliding window on the text [15]. This requires excessive computational resources. ffl Recency rank coding [4,10,29] This is computationally simple but corresponds to a rather coarse model of recency. ffl Exponential aging (giving exponentially increasing weights to successive symbols) 9,20] This is moderately difficult to ....

D. E. Knuth, "Dynamic Huffman Coding," J. Algorithms 6 (June 1985), 163--180. 19

....be determined dynamically, and have the Huffman code evolve over time. Dynamic Huffman coding is the strategy of repeatedly constructing the Huffman code for the input so far, and using it in transmitting the next input symbol. Knuth presented an efficient algorithm for dynamic Huffman coding in [22], and his performance results for the algorithm show it consistently producing compression very near (though not surpassing) the compression attained with static Huffman code for the entire input. Vitter [40, 41] then developed a dynamic Huffman algorithm that improves on Knuth s in the following ....

....adaptive context. For example, adaptive coding algorithms can start at any point in the lattice, as long as both ends of the communication know which one. Rather than rely on the dynamic Huffman algorithm to derive reasonable operating points for the code, or rely on Knuth s windowed algorithm [22], one can immediately begin with a mutually agreed upon, reasonable initial code (depending on the type of information being transmitted) and then adapt this code using some mutually agreed upon greedy algorithm for moving in the imbalance lattice. Acknowledgements. We are very grateful to ....

D.E. Knuth, Dynamic Huffman Coding, J. Algorithms, 6 (1985), pp. 163--180.

....determined dynamically, and have the Huffman code evolve over time. Dynamic Huffman coding is the strategy of repeatedly constructing the Huffman code for the input so far, and using it in transmitting the next input symbol. Knuth presented an efficient algorithm for dynamic Huffman coding in [11], and his performance results for the algorithm show it consistently producing compression very near (though not surpassing) the compression attained with static Huffman code for the entire input. Vitter [24, 25] then developed a dynamic Huffman algorithm that improves on Knuth s in the following ....

D.E. Knuth, "Dynamic Huffman Coding", J. Algorithms 6 (1985), 163--180.

.... sense) for a further simplified, sub optimal family of codes used in practice [15, 5] It should be pointed out that in the adaptive mode, a structured family of codes relaxes the need of dynamically updating code tables due to possible variations in the estimated parameter (see, e.g. [18]) 2 Universal Probability Assignment for TSG s Consider the class of sources defined in (2) where = d) is unknown a priori. Since Rissanen s lower bound on the universal coding redundancy (1) applies (as will be shown in the sequel) and since K = 2, this redundancy essentially cannot ....

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, vol. 6, pp. 163--180, 1985.

....7 Research Issues and Summary The statistical compression algorithm is from Huffman (1951) The UNIX command pack implements the algorithm. The dynamic version has been discovered independently by Faller (1973) and Gallager (1978) Practical versions are given by Cormack and Horspool (1984) and Knuth (1985). A precise analysis leading to an improvement is presented in (Vitter, 1987) The command compact of UNIX implements the dynamic Huffman coding. It is unclear to whom precisely should be attributed the idea of data compression using arithmetic coding. It is sometimes refer to Elias (1963) and ....

Knuth, D.E. 1985. Dynamic Huffman coding. J. Algorithms. 6:163--180.

....universal lossless source coding schemes are usually designed for classes of stationary sources. Not surprisingly, these schemes may perform poorly when the source is non stationary, unless some adaptation mechanism is applied. While adaptive schemes such as the dynamic Huffman code [4] [9], 15] 22] and variations of the sliding window Lempel Ziv algorithm [21] 24] 25] have been developed and applied for general non stationary sources, much less attention has been devoted to systematic, rigorous theoretical development of universal codes for simple classes of non stationary ....

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, Vol. 6, pp. 163-180, June 1985.

.... calculus can be found in [14] 3] 4] 10] 13] and [15] Besides lots of articles on incremental algorithms for specific problems, like for example the algorithm for incrementally computing the minimum spanning tree of Frederickson [7] the algorithm for incremental Huffman coding of Knuth [11], and the algorithm for pattern matching with a dynamically changing set of patterns of Meyer [16] several proposals for the derivation and description of incremental algorithms have been given in the literature. The language INC, designed by Yellin and Strom [25] automatically transforms ....

D.E. Knuth. Dynamic Huffman coding. Journal of Algorithms, 6:163--180, 1985.

No context found.

D. E. Knuth. Dynamic Huffman coding. In Journal of Algorithms,6:163180, June 1985.

No context found.

D. Knuth. Dynamic Huffman Coding. Journal of Algorithms, 6:163--180, 1985.

No context found.

D. Knuth. Dynamic Huffman Coding. Journal of Algorithms, 6:163--180, 1985.

No context found.

D. E. Knuth, "Dynamic Huffman coding," Journal of Algorithms, vol. 6, pp. 163--180, 1985.

No context found.

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, no. 6, pp. 163--180, 1985.

No context found.

D. E. Knuth. "Dynamic Huffman Coding." Journal of Algorithms, Vol. 6, pp. 163-180. 1985.

No context found.

D. E. Knuth. "Dynamic Huffman Coding." Journal of Algorithms, Vol. 6, pp. 163-180. 1985.

No context found.

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, vol. 6, pp. 163--180, 1985.

No context found.

D.E. Knuth, Dynamic Huffman Coding, in: J. Algorithms, 6, 2 (June), 1985, pp. 163180.

No context found.

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, Vol. 6, pp. 163-180, June 1985.

No context found.

D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, vol. 6, pp. 163--180, 1985. 33

No context found.

Knuth, D.E. (1985). Dynamic Huffman coding. Journal of Algorithms, 6, 163-80.

No context found.

Knuth, D.E., "Dynamic Huffman coding," J. Algorithms, 6, pp. 163-180, 1985.

No context found.

Knuth D.E.; "Dynamic Huffman Coding", Journal of Algorithms 6, 163-180.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC