T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-and approach to pattern matching in LZW compressed text. In Proc. 10th Annual Symp. on Combinatorial Pattern Matching (CPM'99), LNCS 1645, pages 1-13. Springer-Verlag, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... text without decoding it, which is often referred to as compressed pattern matching , has been studied extensively [4] 18] 43] Along these lines, string search in compressed text was developed for the compression paradigm of LZ78 [52] and its subsequent variant LZW [50] as described in [30], 44] A more challenging problem is that of fully compressed pattern matching when both the pattern and text strings are compressed [21] 22] For the LZ78 LZW paradigm, compressed matching has been extended and generalized to that of approximate pattern matching ( nding all occurrences of ....

Kida, T., M. Takeda, A. Shinohara, M. Miyazaki, and S. Arikawa, Shift-And approach to pattern matching in LZW compressed text, Proc. 10th Annual Symposium On Combinatorial Pattern Matching, LNCS 1645, 1-13 (1999).

....compressed texts (simple and extended patterns) and specialized it for the particular cases of LZ77, LZ78 and a new variant proposed that was competitive and convenient for search purposes. A similar result, restricted to the LZW format, was independently found and presented by Kida et al. [14]. The same group generalized the existing algorithms and nicely uni ed the concepts in a general framework [12] Recently, Navarro and Tarhio [28] presented a new, faster, algorithm based on Boyer Moore. Approximate string matching on compressed text aims at nding the pattern where a limited ....

T. Kida, M. Takeda, A. Shinohara, M. Miyazaki, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. 10th Annual Symposium on Combinatorial Pattern Matching (CPM'99), LNCS 1645, pages 1-13, 1999.

.... text without decoding it, which is often referred to as compressed pattern matching , has been studied extensively [3] 13] 34] Along these lines, string search in compressed text was developed for the compression paradigm of LZ78 [45] and its subsequent variant LZW [43] as described in [23], 35] A more challenging problem is that of fully compressed pattern matching when both the pattern and text strings are compressed [16] 17] For the LZ78 LZW paradigm, compressed matching has been extended and generalized to that of approximate pattern matching ( nding all occurrences of a ....

Kida, T., M. Takeda, A. Shinohara, M. Miyazaki, and S. Arikawa, Shift-And approach to pattern matching in LZW compressed text, Proc. 10th Annual Symposium On Combinatorial Pattern Matching, LNCS 1645, 1-13 (1999).

....compressed texts (simple and extended patterns) and specialized it for the particular cases of LZ77, LZ78 and a new variant proposed which was competitive and convenient for search purposes. A similar result, restricted to the LZW format, was independently found and presented by Kida et al. [14]. The same group generalized the existing algorithms and nicely unified the concepts in a general framework [12] Recently, Navarro and Tarhio [25] presented a new, faster, algorithm based on Boyer Moore. Approximate string matching on compressed text aims at finding the pattern where a limited ....

T. Kida, M. Takeda, A. Shinohara, M. Miyazaki, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. CPM'99, LNCS 1645, pages 1--13, 1999.

....on Ziv Lempel compressed texts (simple and extended patterns) and specialized it for the particular cases of LZ77, LZ78 and a new variant proposed which was competitive and convenient for search purposes. A similar result, restricted to the LZW format, was independently found and presented in [14]. Finally, 12] generalized the existing algorithms and nicely unified the concepts in a general framework. 3 Basic Concepts 3.1 The Ziv Lempel Compression Formats LZ78 and LZW The general idea of Ziv Lempel compression is to replace substrings in the text by a pointer to a previous occurrence ....

....search methods we have proposed along the paper. As can be seen, BM simple opt is the best choice for natural language, while BM blocks (without the optimization ) is the best on DNA. BM multichar works better than BM simple on DNA, but BM blocks is superior. 1 The bit parallel algorithm of [14] should be similar, but it is implemented over Unix Compress and it is slower. 0.4 0.5 0.6 0.7 0.8 0.9 1 10 20 30 40 50 60 70 80 90 100 m [10 Mb of WSJ] BM simple BM simple opt BM complete BM multichar BM blocks BM blocks opt 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 ....

T. Kida, M. Takeda, A. Shinohara, M. Miyazaki, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. CPM'99, LNCS 1645, pages 1--13, 1999.

....on Ziv Lempel compressed texts (simple and extended patterns) and implemented it for the particular cases of LZ77, LZ78 and a new variant proposed which was competitive and convenient for search purposes. A similar result, restricted to the LZW format, was independently found and presented in [Kida et al. 1999]. Finally, Kida et al. 1999] generalized the existing algorithms and nicely unified the concepts in a general framework. All the empirical results obtained roughly coincide in a general figure: searching on a Ziv Lempel compressed text can take half the time of decompressing that text and then ....

....texts (simple and extended patterns) and implemented it for the particular cases of LZ77, LZ78 and a new variant proposed which was competitive and convenient for search purposes. A similar result, restricted to the LZW format, was independently found and presented in [Kida et al. 1999] Finally, [Kida et al. 1999] generalized the existing algorithms and nicely unified the concepts in a general framework. All the empirical results obtained roughly coincide in a general figure: searching on a Ziv Lempel compressed text can take half the time of decompressing that text and then searching it. However, the ....

Kida, T., Takeda, M., Shinohara, A., Miyazaki, M., and Arikawa, S. 1999. Shift-And approach to pattern matching in LZW compressed text. In Proc. 10th Annual Symposium on Combinatorial Pattern Matching (CPM'99), LNCS 1645 (1999), pp. 1--13.

....on Ziv Lempel compressed texts (simple and extended patterns) and specialized it for the particular cases of LZ77, LZ78 and a new variant proposed which was competitive and convenient for search purposes. A similar result, restricted to the LZW format, was independently found and presented in [12]. In [17] a new, faster, algorithm was presented based on Boyer Moore. The aim of this paper is to present a general solution to the approximate string matching problem on compressed text in the LZ78 and LZW formats. 3 Approximate String Matching by Dynamic Programming We introduce some ....

T. Kida, M. Takeda, A. Shinohara, M. Miyazaki, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. CPM'99, LNCS 1645, pages 1--13, 1999.

....However, as we show in the experiments, the performance does not improve. 5 LZ77 Compression 5. 1 Compression Algorithm The Ziv Lempel compression algorithm of 1977 (usually named LZ77 [31] is, in some sense, simpler than LZ78, since the basic idea is just to recognize two 2 See, however, [18], in this very same conference. repeated segments of the text and to mark the second as a reference (position in the text and length of the repeated part) to the first one. More formally, assume that a prefix t 1 : t i of T has been already compressed in a sequence of blocks Z = b 1 : b ....

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-and approach to pattern matching in lzw compressed text. In Proc. CPM'99, 1999. To appear.

No context found.

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. 10th Ann. Symp. on Combinatorial Pattern Matching, Lecture Notes in Computer Science. Springer-Verlag, 1999. to appear.

No context found.

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. 10th Ann. Symp. on Combinatorial Pattern Matching, pages 1--13. Springer-Verlag, 1999.

....et al. 26]presented an algorithm based on a similar technique for the compression using anti dictionaries. As other works on the Ziv Lempel family, Farach and Thorup[11] and Gtsieniec, et al. 13] addressed the LZ77 compression. Bit parallel realization of [4] was inde pendently proposed in [17, 22] and proved to be fast in practice for a short pattern. Recently, new practical results appeared. Miyazaki, et al. 21] addressed the Huffman encoding. Moura, et al. 9, 10] addressed a new compression scheme that uses a word based Huffman encoding with a byte oriented code. Shibata, et al. 24, ....

....scheme that uses a word based Huffman encoding with a byte oriented code. Shibata, et al. 24, 25] addressed the byte pair encoding [12] which is a simple version of the RE PAIR. Their algorithms run even faster than pattern matching in uncompressed texts. This paper is based on [18] and [17]. Original text Compressed text b 1,2, 4, 4, 5, 2, 3, 6, 9, 11 FIG. 1. Dictionary trie. 3 Preliminaries In the following subsections we briefly sketch the LZW compression, and review the Aho Corasick pattern matching machine and the generalized suffix trie[15] These data structure are used ....

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. loth Ann. Symp. on Combinatorial Pattern Matching, Lecture Notes in Computer Science, pages 1-13. Springer-Verlag, 1999.

No context found.

T. Kida, M. Takeda, A. Shinohara, M. Miyazaki, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. CPM'99, LNCS 1645, pages 1--13, 1999.

....text. As we have shown, the collage systems for Sequitur and LZW are truncation free. On the other hand, the collage systems for LZ77 and LZSS compression requires truncations [8] These facts correspond with the observations that LZW is suitable for compressed pattern matching, while LZSS is not [7, 8, 16]. As we will show below, Sequitur is also suitable for compressed pattern matching. 3.3 Searching in Sequitur compressed files We briefly state on the modification of the general pattern matching algorithm that are specific to Sequitur. In Sequitur, the encoding of the dictionary D is ....

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. 10th Ann. Symp. on Combinatorial Pattern Matching, pages 1--13. Springer-Verlag, 1999.

....the complexity of this problem for various compression methods from the viewpoint of combinatorial pattern matching. It is theoretically interesting, and in practice some algorithms proposed are indeed faster than a regular decompression followed by a simple search. In fact, Kida et al. [19, 18] and Navarro et al. 22] independently presented compressed pattern matching algorithms for the Lempel Ziv Welch (LZW) compression which run faster than a decompression followed by a search. However, the algorithms are slow in comparison with pattern matching in uncompressed text. In other words, ....

....The array size is not critical since the number of phrases in D is at most 256 in BPE compression. This is not the case with LZW, in which jDj can be the compressed text size. Another implementation is the one utilizing the bit parallel paradigm in a similar way that we did for LZW compression [18]. Technical details are omitted because of lack of space. 4 Experimental results We estimated the running time of the proposed algorithms running on BPE compressed files. We tested the two implementations mentioned in the previous section. For comparisons, we tested the 7 Table 3: Performance ....

[Article contains additional citation context not shown here]

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. 10th Ann. Symp. on Combinatorial Pattern Matching, pages 1--13. Springer-Verlag, 1999. 9

....occurrences. We implemented a simple version of the algorithm and observed that it is approximately twice faster than a decompression followed by a search using the Aho Corasick automaton. We took another implementation of the algorithm utilizing bitparallelism, and reported some experiments [10]. Independently, Navarro and Raffinot [14] developed a more general technique for string matching on a text given as a sequence of blocks, which abstracts both LZ77 and LZ78 compressions, and gave bit parallel implementations. The running time of these algorithms based on the bit parallelism for ....

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. ShiftAnd approach to pattern matching in LZW compressed text. In Proc. 10th Ann. Symp. on Combinatorial Pattern Matching, Lecture Notes in Computer Science. Springer-Verlag, 1999. to appear.

....A. Amir, G. Benson, and M. Farach [6] LZ77 M. Farach and M. Thorup [17] L. G asieniec, M. Karpinski, W. Plandowski, and W. Rytter [21] LZW A. Amir, G. Benson, and M. Farach [5] T. Kida, M. Takeda, A. Shinohara, M. Miyazaki, and S. Arikawa [26] T. Kida, M. Takeda, A. Shinohara, and S. Arikawa [25]; G. Navarro and M. Raffinot [33] straight line program M. Karpinski, W. Rytter, and A. Shinohara [24] M. Miyazaki, A. Shinohara, and M. Takeda [32] Huffman S. Fukamachi, T. Shinohara, and M. Takeda [19] M. Miyazaki, S. Fukamachi, M. Takeda, and T. Shinohara [31] finite state encoding M. Takeda ....

....a two dimensional array since the number of different codes is at most 256 in BPE. This is not the case with LZW, in which the number of codes can be the compressed text size. Alternative implementation is the one utilizing the bit parallel paradigm in a similar way that we did for LZW compression [25] (technical details are omitted) 4 Experimental results We estimated the running time of the algorithms presented in the previous section in searching BPE compressed text, in comparison with those of the algorithm for searching LZW compressed text [25] and ordinary algorithms searching ....

[Article contains additional citation context not shown here]

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. 10th Ann. Symp. on Combinatorial Pattern Matching, Lecture Notes in Computer Science. Springer-Verlag, 1999. to appear.

....occurrences. We implemented a simple version of the algorithm and observed that it is approximately twice faster than a decompression followed by a search using the Aho Corasick automaton. We took another implementation of the algorithm utilizing bit parallelism, and reported some experiments [7]. Independently, Navarro and Ra#not [11] developed a more general technique for string matching on a text given as a sequence of blocks, which abstracts both LZ77 and LZ78 compressions, and gave bit parallel implementations. The running time of these algorithms based on the bit parallelism for LZW ....

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. In Proc. 10th Ann. Symp. on Combinatorial Pattern Matching, Lecture Notes in Computer Science. SpringerVerlag, 1999. to appear.

....proposed recently Crochemore et al. 8] We presented an algorithm which has a linear time complexity proportional to the compressed text length, when we exclude the pattern preprocessing. We are now implementing the algorithm to evaluate its performance from practical viewpoints. In [14] we showed that the Shift And approach is effective in the compressed pattern matching for the LZW compression. We think that the Shift And approach will be substituted for the KMP automaton approach presented in this paper and show a good performace in practice when the pattern length m is not so ....

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-And approach to pattern matching in LZW compressed text. Technical Report DOI-TR-CS-156, Department of Informatics, Kyushu University, January 1999.

No context found.

T. Kida, M. Takeda, A. Shinohara, and S. Arikawa. Shift-and approach to pattern matching in LZW compressed text. In Proc. 10th Annual Symp. on Combinatorial Pattern Matching (CPM'99), LNCS 1645, pages 1-13. Springer-Verlag, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC