Finding similar files in a large file system

DMCA

Finding similar files in a large file system (1994)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Udi Manber , Udi Manber

Venue:	in Proceedings of the USENIX Winter 1994 Technical Conference
Citations:	260 - 3 self

BibTeX

@INPROCEEDINGS{Manber94findingsimilar,
    author = {Udi Manber and Udi Manber},
    title = {Finding similar files in a large file system},
    booktitle = {in Proceedings of the USENIX Winter 1994 Technical Conference},
    year = {1994},
    pages = {1--10}
}

OpenURL

Abstract

We present a tool, called sif, for finding all similar files in a large file system. Files are considered similar if they have significant number of common pieces, even if they are very different otherwise. For example, one file may be contained, possibly with some changes, in another file, or a file may be a reorganization of another file. The run-ning time for finding all groups of similar files, even for as little as 25 % similarity, is on the order of 500MB to 1GB an hour. The amount of similarity and several other customized parameters can be determined by the user at a post-processing stage, which is very fast. Sif can also be used to very quickly identify all similar files to a query file using a preprocessed index. Application of sif can be found in file management, information collecting (to remove duplicates), program reuse, file synchronization, data compression, and maybe even plagiarism detection. 1.

DMCA