CoralCDN is a peer-to-peer content distribution network that allows a user to run a web site that offers high performance and meets huge demand, all for the price of a cheap broadband Internet connection. Volunteer sites that run CoralCDN automatically replicate content as a side effect of users accessing it. Publishing through CoralCDN is as simple as making a small change to the hostname in an object's URL; a peer-to-peer DNS layer transparently redirects browsers to nearby participating cache nodes, which in turn cooperate to minimize load on the origin web server. One of the system's key goals is to avoid creating hot spots that might dissuade volunteers and hurt performance. It achieves this through Coral, a latency-optimized hierarchical indexing infrastructure based on a novel abstraction called a distributed sloppy hash table, or DSHT.

It is becoming increasingly common to construct network services using redundant resources geographically distributed across the Internet. Content Distribution Networks are a prime example. Such systems distribute client requests to an appropriate server based on a variety of factors---e.g., server load, network proximity, cache locality---in an effort to reduce response time and increase the system capacity under load. This paper explores the design space of strategies employed to redirect requests, and defines a class of new algorithms that carefully balance load, locality, and proximity. We use large-scale detailed simulations to evaluate the various strategies. These simulations clearly demonstrate the effectiveness of our new algorithms, which yield a 60-91% improvement in system capacity when compared with the best published CDN technology, yet user-perceived response latency remains low and the system scales well with the number of servers.

The Domain Name System (DNS) is a ubiquitous part of everyday computing, translating human-friendly machine names to numeric IP addresses. Most DNS research has focused on server-side infrastructure, with the assumption that the aggressive caching and redundancy on the client side are sufficient. However, through systematic monitoring, we find that client-side DNS failures are widespread and frequent, degrading DNS performance and reliability. We introduce CoDNS, a lightweight, cooperative DNS lookup service that can be independently and incrementally deployed to augment existing nameservers. It uses a locality and proximity-aware design to distribute DNS requests, and achieves low-latency, low-overhead name resolution, even in the presence of local DNS nameserver delay/failure. Using live traffic, we show that CoDNS reduces average lookup latency by 27-82%, greatly reduces slow lookups, and improves DNS availability by an additional ’9’. We also show that a widely-deployed service using CoDNS gains increased capacity, higher reliability, and faster start times. 1

Scalable distribution of large files has been the area of much research and commercial interest in the past few years. In this paper, we describe the CoBlitz system, which efficiently distributes large files using a content distribution network (CDN) designed for HTTP. As a result, CoBlitz is able to serve large files without requiring any modifications to standard Web servers and clients, making it an interesting option both for end users as well as infrastructure services. Over the 18 months that CoBlitz and its partner service, CoDeploy, have been running on PlanetLab, we have had the opportunity to observe its algorithms in practice, and to evolve its design. These changes stem not only from observations on its use, but also from a better understanding of their behavior in real-world conditions. This utilitarian approach has led us to better understand the effects of scale, peering policies, replication behavior, and congestion, giving us new insights into how to better improve their performance. With these changes, CoBlitz is able to deliver in excess of 1 Gbps on PlanetLab, and to outperform a range of systems, including research systems as well as the widely-used BitTorrent. 1

A Content Discovery System (CDS) allows nodes in the system to discover contents published by some other nodes in the system. Existing CDS systems have difficulties in achieving both scalability and rich functionality. In this paper, we present the design and evaluation of a distributed and scalable CDS. Our system uses Rendezvous Points (RPs) for content registration and query resolution, and can accommodate frequent updates from dynamic contents. Contents stored in our system can be searched via subset matching. We propose a novel mechanism that uses load balancing matrices (LBMs) to dynamically balance both registration and query load across nodes in the system to maintain high system throughput even under skewed load. Our system utilizes existing Distributed Hash Table (DHT) mechanisms for CDS overlay network management and routing. We validate our system's scalability and load balancing properties using extensive simulation.

ABSTRACT: Hierarchies provide scalability in large net-works and are integral to many widely-used protocols and applications. Previous approaches to constructing hierar-chies have typically either assumed static hierarchy configu-ration, or have used bottom-up construction methods. We describe how to construct hierarchies in a top-down fash-ion, and show that our method is much more efficient than bottom-up methods. We also show that top-down hierarchy construction is a better choice when administrative policy constraints are imposed on hierarchy formation. 1

Establishing pairwise keys for each pair of neighboring sensors is the first concern in securing communication in sensor networks. This task is challenging because resources are limited. Several random key predistribution schemes have been proposed, but they are appropriate only when sensors are uniformly distributed with high density. These schemes also suffer from a dramatic degradation of security when the number of compromised sensors exceeds a threshold. In this paper, we present a group-based key predistribution scheme, GKE, which enables any pair of neighboring sensors to establish a unique pairwise key, regardless of sensor density or distribution. Since pairwise keys are unique, security in GKE degrades gracefully as the number of compromised nodes increases. In addition, GKE is very efficient since it requires only localized communication to establish pairwise keys, thus significantly reducing the communication overhead. Our security analysis and performance evaluation illustrate the superiority of GKE in terms of resilience, connectivity, communication overhead and memory requirement. Categories and Subject Descriptors C.2 [Computer-Communication Networks]: secuirty and protection;

A novel scheme for processing packets in a router is presented, which provides for load sharing among multiple network processors distributed within the router. It is complemented by a feedback control mechanism designed to prevent processor overload. Incoming traffic is scheduled to multiple processors based on a deterministic mapping. The mapping formula is derived from the robust hash routing (also known as the highest random weight - HRW) scheme, introduced in K.W. Ross, IEEE Network, 11(6), 1997, and D.G. Thaler et al., IEEE Trans. Networking, 6(1), 1998. No state information on individual flow mapping needs to be stored, but for each packet, a mapping function is computed over an identifier vector, a predefined set of fields in the packet. An adaptive extension to the HRW scheme is provided in order to cope with biased traffic patterns. We prove that our adaptation possesses the minimal disruption property with respect to the mapping and exploit that property in order to minimize the probability of flow reordering. Simulation results indicate that the scheme achieves significant improvements in processor utilization. A higher number of router interfaces can thus be supported with the same amount of processing power. I.

We present a troubleshooting approach for coordinating problem diagnosis, and describe Global Distributed Troubleshooting (GDT), a distributed protocol which realizes this approach. We show through simulation that GDT scales well as the number of observers and problems grows.

We present a multicast routing protocol called Distributed Core Multicast (DCM). It is intended for use within a large single Internet domain network with a very large number of multicast groups with a small number of receivers. Such a case occurs, for example, when multicast addresses are allocated to mobile hosts, as a mechanism to manage Internet host mobility or in large distributed simulations. For such cases, existing dense or sparse mode multicast routing algorithms do not scale well with the number of multicast groups. DCM is based on an extension of the centre-based tree approach. It uses several core routers, called Distributed Core Routers (DCRs) and a special control protocol among them. DCM aims: (1) avoiding multicast group state information in backbone routers, (2) avoiding triangular routing across expensive backbone links, (3) scaling well with the number of multicast groups. We evaluate the performance of DCM and compare it to an existing sparse mode routing protocol when there is a large number of small multicast groups. We also analyse the behaviour of DCM when the number of receivers per group is not a small number.