Fraenkel A.S., Klein S.T., Novel Compression of Sparse Bit-Strings, in Combinatorial Algorithms on Words, NATO ASI Series Vol F12, Springer Verlag, Berlin (1985) 169--183.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....i.e. there 3 are no three integers i j such that n i 6= 0; n 6= 0, but n j = 0. This is true for many real life distributions, and in particular for all the examples below. On the other hand, the distribution of one of the alphabets used for compressing a set of sparse bitmaps in [8] is h1; 0; 0; 1; 7; 0; 1; 28; 0; 46; 59; 114i. All the techniques suggested herein can be easily adapted to the general case using a vector succ(i) giving for each codeword length i, the next larger codeword length j for which n j 0. But to make the exposition clearer, we shall suppress ....

Fraenkel A.S., Klein S.T., Novel Compression of Sparse Bit-Strings, in Combinatorial Algorithms on Words, NATO ASI Series Vol F12, Springer Verlag, Berlin (1985) 169--183.

....two or more could share the same column, yielding a new bit matrix with a very high density of 1 s. This matrix can therefore be compressed efficiently, for example by complementing each column and then using one of the known techniques for compressing sparse vectors, e.g. Teuhola, 1978) or (Fraenkel and Klein, 1985). Thus we retain the bitmap approach, but at less cost than actually increasing the value of k. The scheme of Section 2 seems to be best suited for the words of the third class, the intermediate range, which, when stop words are ignored, account for the large majority of entries in the ....

Fraenkel A.S., Klein S.T., (1985). Novel compression of sparse bit-strings, Combinatorial Algorithms on Words, NATO ASI Series Vol F12, Springer Verlag, Berlin 169--183.

....itself is encoded as a sequence of such codewords. For sparse vectors, the k bit block consisting of zeros only, and blocks with only a single 1 bit, have much higher probabilities than the other blocks, so the average codeword length of the Huffman code will be smaller than k. Fraenkel Klein [7] combine Huffman coding with run length coding. Once again, a parameter k is chosen as a block size. However, since for very sparse vectors the probability of a block of k zeros is high, runs of blocks of k zeros receive special treatment. We first represent the succession of k bit blocks ....

Fraenkel A.S., Klein S.T., Novel Compression of sparse Bit-Strings, in Combinatorial Algorithms on Words, NATO ASI Series Vol F12, Springer Verlag, Berlin (1985) 169--183.

....for 21 v 1 , forming v 2 , and so on. At each stage, when storing v i , all blocks corresponding to 0 s in v i 1 are dropped. The method is improved in [4] by pruning as well some of the branches of the hierarchy which ultimately point to very few 1 bits. A different method suggested in [16] combines Huffman coding with run length coding for blocks of zeros. The methods in [4] and [16] yield compression of up to 94 on a set of bitmaps constructed at RRP. A different kind of bit map file is a so called signature file (see for example Faloutsos Christodoulakis [12] Here the text ....

....to 0 s in v i 1 are dropped. The method is improved in [4] by pruning as well some of the branches of the hierarchy which ultimately point to very few 1 bits. A different method suggested in [16] combines Huffman coding with run length coding for blocks of zeros. The methods in [4] and [16] yield compression of up to 94 on a set of bitmaps constructed at RRP. A different kind of bit map file is a so called signature file (see for example Faloutsos Christodoulakis [12] Here the text is partitioned into relatively small parts P , each of which is assigned a signature, which is a ....

[Article contains additional citation context not shown here]

Fraenkel A.S., Klein S.T., Novel compression of sparse bit-strings, Combinatorial Algorithms on Words, NATO ASI Series Vol F12, Springer Verlag, Berlin (1985) 169--183.

....and extends preliminary versions that were presented at the DCC 91 Conference in Snowbird, Utah, and at the SIGIR 91 Conference in Chicago. 1 rates of up to 95 have been reported. Techniques for compressing bitmaps include variants of run length coding [17] 18] Huffman coding [12] [10], and hierarchical methods [19] 6] All of these try to compress each bitmap independently from the others. In [3] a new method is introduced that attempts to improve compression efficiency of a set of bitmaps by collecting similar maps into clusters. In the current work, we try a different ....

....power less so, such a tradeoff may well be acceptable. A final concern is the occurrence of probabilities near one for zero blocks in some regions of the table. This might yield innefficient Huffman codes. A possibility, within the Huffman framework, is to use variable length blocks as in [10], but with the block length being calculated from the model s parameters. Appendix: Comment on binomial coefficients Some readers may be struck by the deviation of our formula for i m k j from other, more customary ones when k 0. But there is justification for this deviation beyond the fact ....

Fraenkel A.S., Klein S.T., Novel compression of sparse bit-strings, in Combinatorial Algorithms on Words, NATO ASI Series Vol F12, Springer Verlag, Berlin (1985) 169--183.

....block of 8 consecutive zeros has probability 0.925 (it is larger than (1 Gamma 0:017) 8 because the 1 bits are not uniformly scattered through the maps) The next step consists therefore of generating also codewords for runs of 0 blocks of various lengths. Several such methods are suggested in [15]. The results of methods POW2 and LLRUN of [15] appear in the third and fourth lines of Table 2; the benefit of arithmetic codes has now been reduced to merely half a percent. Noting the effect of the EOF requirement of arithmetic codes further reduces the 8 Huffman cost. In our case, each ....

....(it is larger than (1 Gamma 0:017) 8 because the 1 bits are not uniformly scattered through the maps) The next step consists therefore of generating also codewords for runs of 0 blocks of various lengths. Several such methods are suggested in [15] The results of methods POW2 and LLRUN of [15] appear in the third and fourth lines of Table 2; the benefit of arithmetic codes has now been reduced to merely half a percent. Noting the effect of the EOF requirement of arithmetic codes further reduces the 8 Huffman cost. In our case, each bitmap has to be accessible individually. ....

Fraenkel A.S., Klein S.T., Novel compression of sparse bit-strings, Combinatorial Algorithms on Words, NATO ASI Series Vol F12, Springer Verlag, Berlin (1985) 169--183.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC