During recent years, Distributed Hash Tables (DHTs) have been extensively studied through simulation and analysis. However, due to their limited deployment, it has not been possible to observe the behavior of a widely-deployed DHT in practice. Recently, the popular eMule file-sharing software incorporated a Kademlia-based DHT, called Kad, which currently has around one million simultaneous users. In this paper, we empirically study the performance of the key DHT operation, lookup, over Kad. First, we analytically derive the benefits of different ways to increase the richness of routing tables in Kademlia-based DHTs. Second, we empirically characterize two aspects of the accuracy of routing tables in Kad, namely completeness and freshness, and characterize their impact on Kad’s lookup performance. Finally, we investigate how the efficiency and consistency of lookup in Kad can be improved by performing parallel lookup and maintaining multiple replicas, respectively. Our results pinpoint the best operating point for the degree of lookup parallelism and the degree of replication for Kad.

Abstract—In the past few years, peer-to-peer (P2P) networks have become a promising paradigm for building a wide variety of distributed systems and applications. The most popular P2P application till today is file sharing, e.g., Gnutella, Kazza, etc. These systems are usually referred to as unstructured, and search in unstructured P2P networks usually involves flooding or random walking. On the other hand, in structured P2P networks (DHTs), search is usually performed by looking up a distributed inverted index. The efficiency of the search mechanism is the key to the scalability of a P2P content sharing system. So far, neither unstructured nor structured P2P networks alone can solve the search problem in a satisfactory way. In this paper, we propose to combine the strengths of both unstructured and structured P2P networks to achieve more efficient search. Specifically, we propose to enhance search in unstructured P2P overlay networks by building a partial index of shared data using a structured P2P network. The index maintains two types of information: the top interests of peers and globally unpopular data, both characterized by data properties. The proposed search protocol, assisted search with partial indexing, makes use of the index to improve search in three ways: First, the index assists peers to find other peers with similar interests and the unstructured search overlay is formed to reflect peer interests. Second, the index also provides search hints for those data difficult to locate by exploring peer interest locality, and these hints can be used for second-chance search. Third, the index helps to locate unpopular data items. Experiments based on a P2P file sharing trace show that the assisted search with a lightweight partial indexing service can significantly improve the success rate in locating data than Gnutella and a hit-rate-based protocol in unstructured P2P systems, while incurring low search latency and overheads. Index Terms—Distributed systems, information search and retrieval. 1

The dynamics of peer participation, or churn, are an inherent property of Peer-to-Peer (P2P) systems that should be incorporated in both the design and evaluation of P2P systems. This requires a proper characterization of churn in real-world P2P networks. However, the few previous measurement-based studies on the characterization of P2P systems have used either unrepresentative group of peers or coarse-grain measurements. In this extended abstract, we characterize churn in the Gnutella network based on fine-grained monitoring of the entire population. We developed a new crawler that can capture a complete snapshot of Gnutella network within a few minutes. This not only improves the accuracy, by reducing distortion in captured snapshots, but it also increase the granularity of captured dynamics. We present our preliminary characterizations of peer uptime and discuss their implications. In particular, we show that peer uptime follows a power-law distribution rather than the commonly assumed Poisson distribution. In a nutshell, a large portion of up peers are highly stable, yet the remaining peers turnover very quickly.

Traditional peer-to-peer (p2p) applications such as Kazaa and Gnutella have been primarily used for sharing read-only files (such as mpegs, jpegs, and mp3s) over the Internet. Such applications did not demand mechanisms to protect concurrent access and update to shared data, and therefore mutual exclusion has not yet been thoroughly studied in the p2p community. Due to the recent surge in the area of Grid computing, there is an urgency to find efficient ways of sharing resources. In the future, application developers should be able to write Grid and p2p applications without worrying about handling the low-level details of protecting consistent and concurrent access to shared resources. This thesis proposes two novel protocols for achieving mutual exclusion efficiently in dynamic p2p systems. The protocols are layered atop a distributed hash table (DHT), making them scalable and fault-tolerant. The burden of controlling access to the critical section is also evenly distributed among all the nodes in the network, making the protocols more distributed and easily adaptable to growing networks. Since the protocols are designed independent of any specific DHT implementation, they can be incorporated

Abstract. In this paper we present a scalable distributed ontology storage system (OntStore), which unlike other ontology repositories is capable of storing and querying the ontologies and the resource descriptions in a distributed manner. It is based on the distributed hash table (DHT) approach emerging in the P2P arena and well known W3C standardized languages. The system decomposes the ontologies into the corresponding triples (subject, predicate, object) and uses the DHT system called Pastry to hash and store the elements. There are no super-peers in the system and both exact-matched and range queries can be routed. The system ensures, that queries do find the resources if the resources are available in the network. It also guaranties, that the number of routing hops for inserting a resource and resolving most queries is logarithmic to the number of nodes in the network. 1