Reclaiming Space from Duplicate Files in a Serverless Distributed File System

Reclaiming Space from Duplicate Files in a Serverless Distributed File System (2002)

by John R. Douceur , Atul Adya , William J. Bolosky , Dan Simon , Marvin Theimer

Venue:	In Proceedings of 22nd International Conference on Distributed Computing Systems (ICDCS
Citations:	43 - 2 self

Datum	Value	Source
TITLE	Reclaiming Space from Duplicate Files in a Serverless Distributed File System	SVM HeaderParse 0.1
AUTHOR NAME	John R. Douceur	SVM HeaderParse 0.1
AUTHOR NAME	Atul Adya	SVM HeaderParse 0.1
AUTHOR NAME	William J. Bolosky	SVM HeaderParse 0.1
AUTHOR NAME	Dan Simon	SVM HeaderParse 0.1
AUTHOR NAME	Marvin Theimer	SVM HeaderParse 0.1
ABSTRACT	The Farsite distributed file system provides availability by replicating each file onto multiple desktop computers. Since this replication consumes significant storage space, it is important to reclaim used space where possible. Measurement of over 500 desktop file systems shows that nearly half of all consumed space is occupied by duplicate files. We present a mechanism to reclaim space from this incidental duplication to make it available for controlled file replication. Our mechanism includes 1) convergent encryption, which enables duplicate files to coalesced into the space of a single file, even if the files are encrypted with different users ’ keys, and 2) SALAD, a Self-Arranging, Lossy, Associative Database for aggregating file content and location information in a decentralized, scalable, fault-tolerant manner. Large-scale simulation experiments show that the duplicate-file coalescing system is scalable, highly effective, and fault-tolerant. 1.	SVM HeaderParse 0.1
CITATIONS	38 found	ParsCit 1.0