Citation Context

...of ongoing research projects are currently exploring this problem. 11 projects have proposed systems that support similar hash table functionality; among them are Tapestry [50], Pastry [39] and Chord =-=[47]-=-. We will use the term Distributed Hash Table or DHT to refer to the above systems (including CAN) collectively. In all these DHT systems,sles are associated with a key (produced, for instance, by has...

Citation Context

...vice. A number of ongoing research projects are currently exploring this problem. 11 projects have proposed systems that support similar hash table functionality; among them are Tapestry [50], Pastry =-=[39]-=- and Chord [47]. We will use the term Distributed Hash Table or DHT to refer to the above systems (including CAN) collectively. In all these DHT systems,sles are associated with a key (produced, for i...

Citation Context

...d 2ms for intra-stub domain links (we also experimented with a delay distribution of 100, 10 and 1ms instead of 20, 5 and 2ms respectively with no real change in our results). 2. PLRG: Recent studies =-=[12, 48]-=- have indicated that the Internet's degree distribution follows a power-law. Motivated by these observations, degree-based generators have been proposed [1] which appear to better model the measured I...

Citation Context

...or many years now. Yet, IP multicast deployment has been slowed 57 by dicult issues related to scalable inter-domain routing protocols, charging models, robust congestion control schemes and so forth =-=[13, 19, 28]-=-. Because of the problems facing the deployment of a network-level multicast service, many recent research proposals have argued for an application-level multicast service [4, 13, 19] as a more tracta...

Citation Context

...ch lookup service. A number of ongoing research projects are currently exploring this problem. 11 projects have proposed systems that support similar hash table functionality; among them are Tapestry =-=[50]-=-, Pastry [39] and Chord [47]. We will use the term Distributed Hash Table or DHT to refer to the above systems (including CAN) collectively. In all these DHT systems,sles are associated with a key (pr...

Citation Context

...paper, propose a particular CAN design. However, the applicability of CANs is not limited to peer-to-peer systems. CANs could also be used in large scale storage management systems such as OceanStore =-=[15]-=-, Farsite [3], and Publius [18]. These systems all require efficient insertion and retrieval of content in a large distributed storage infrastructure, and a scalable indexing mechanism is an essential...

Citation Context

...orithm When a node fails, one of its neighboring nodes “takes over” the failed node’s now vacant sub-space. Our algorithms draw heavily from the soft-state [21] style recovery algorithms described in =-=[6]-=-. Nodes send periodic update messages to their neighbors. These periodic message updates include the node’s virtual coordinates, and its list of neighbors (both their IP addresses and coordinate sub-s...

Citation Context

...e distribution, Internet TV, video conferencing and shared white-boards [29, 20, 27, 23] require one-to-many message transmission to enable ecient many-to-many communication. The IP Multicast service =-=[10]-=- was proposed as an extension to the Internet architecture to support ecient multi-point packet delivery at the network level. With IP Multicast, a single packet transmitted at the source is delivered...

Citation Context

...xing schemes. Although many of these systems address additional, related problems such as security, anonymity, keyword searching etc, we focus here on their solutions to the indexing problem. Freenet =-=[10, 4]-=- is a file sharing application that additionally protects the anonymity of both authors and readers. Freenet nodes hold 3 types of information: keys (which are analogous to web URLs) addresses of othe...

Citation Context

..., is already proving to be a useful substrate for large distributed systems; a number of projects are proposing to build Internet-scale facilities layered above DHTs, including distributedsle systems =-=[11, 25, 9-=-], application-layer multicast [36, 51], event notication services [3, 40], and chat services [2]. With so many applications being developed in so short a time, we expect the DHT functionality to beco...

Citation Context

...rience, and know the application requirements better, we are not prepared to decide on these tradeoffs. We simulated our CAN design on Transit-Stub (TS) topologies using the GT-ITM topology generator =-=[26]-=-. TS topologies model networks using a 2-level hierarchy of routing domains with transit domains that interconnect lower level stub domains. 3.1 Multi-dimensioned Coordinate Spaces The first observati...

Citation Context

... not well-suited for our application. Because the reasons involve the details of the algorithm, we discuss them at some length in Appendix C. Algorithms built around the concept of geographic routing =-=[14, 17] are-=- similar to our CAN routing algorithm in that they build around the notion of forwarding messages through a coordinate space. The key difference is that the “space” in their work refers to true ph...

Citation Context

...sh structure. The resultant protocols are more complex because the tree-rst approach results in expensive loop detection and avoidance techniques and must be made resilient to 72 partitions. Overcast[=-=21-=-] is a scheme for source-specic, reliable multicast using an overlay network. Overcast constructs ecient dissemination trees rooted at the single source of trac. The overlay network in Overcast is com...

Citation Context

...ing mechanism is an essential component. In fact, as we discuss in Section 5, the OceanStore system already includes a CAN in its core design (although the OceanStore CAN, based on Plaxton’s algorit=-=hm[21]-=-, is somewhat different from what we propose here). Another potential application for CANs is in the construction of wide-area name resolution services that (unlike the DNS) decouple the naming scheme...

Citation Context

...pace is dynamically partitioned among all the nodes in the system such that every node \owns" its individual, distinct zone within the overall space. For example, Figure 2.1 shows the a 2-dimensi=-=onal [0; 1-=-] [0; 1] coordinate space with 5 nodes. Nodes in the CAN self-organize into an overlay network that represents this virtual coordinate space. A node learns and maintains as its set of neighbors the I...

Citation Context

...orwarded without requiring a routing protocol to explicitly construct distribution trees. Finally, M-CAN chooses to extend the basic CAN routing algorithm. An alternate approach, recently proposed in =-=[46]-=-, is to leave the underlying routing algorithm unchanged and provide multicast functionality using only the DHT lookup operation. Understanding the relative advantages and disadvantages of the two app...

Citation Context

...ly manner. The JXTA group at SUN Microsystems is building a CAN-based Chat service [2] that uses DHTs to locate resources such as person identiers or group names. The Geographic Hash Table (GHT) [44=-=, 34]-=- is a DHT for large-scale networks of sensors. Although functionally equivalent to DHTs, GHT does not directly adopt the routing algorithms of the DHTs (CAN, Chord, etc.) but instead uses GPSR [22], a...

Citation Context

...rate for large distributed systems; a number of projects are proposing to build Internet-scale facilities layered above DHTs, including distributedsle systems [11, 25, 9], application-layer multicast =-=[36, 51-=-], event notication services [3, 40], and chat services [2]. With so many applications being developed in so short a time, we expect the DHT functionality to become an integral part of the future P2P ...

Citation Context

...ally obey the triangle inequality. However, we do not yet know how to realistically model the latencies on a PLRG. 54 A detailed evaluation of distributed binning and its applications is presented in =-=[37-=-]. Here, we restrict ourselves to evaluating the performance of CAN with binning-based construction. Figure 3.4 plots the CAN latency stretch (dened above) for increasing CAN sizes, i.e. for increasin...

Citation Context

... truly distributed routing algorithm, both because this does not stress a small set of nodes and it avoids a single point of failure. We hence avoided more traditional hierarchical routing algorithms =-=[2, 24, 16, 6]. Pe-=-rhaps closest in spirit to the CAN routing scheme is the Plaxton algorithm [21]. In Plaxton’s algorithm, every node is assigned a unique n bit label. This � bit label is divided into l levels, wit...

Citation Context

... a particular CAN design. However, the applicability of CANs is not limited to peer-to-peer systems. CANs could also be used in large scale storage management systems such as OceanStore [15], Farsite =-=[3]-=-, and Publius [18]. These systems all require efficient insertion and retrieval of content in a large distributed storage infrastructure, and a scalable indexing mechanism is an essential component. I...

Citation Context

...discovers the IP address of any node currently in the system. The functioning of a CAN does not depend on the details of how this is done, but we use the same bootstrap mechanism as Yallcast and YOID =-=[8]-=-. As in [8] we assume that a CAN has an associated DNS domain name, and that this resolves to the IP address of one or more CAN bootstrap nodes. A bootstrap node maintains a partial list of CAN 3 Seve...

Citation Context

...ring no communication or cooperation between the nodes being binned). Distributed binning is however, just one of several conceivable approaches to acquiring topological information (see for example, =-=[14, 32, 5, 24-=-]). We use distributed binning because it is simple and lightweight enough to be of practical use in very large scale distributed applications; however any similarly scalable technique that oers reaso...

Citation Context

... design. However, the applicability of CANs is not limited to peer-to-peer systems. CANs could also be used in large scale storage management systems such as OceanStore [15], Farsite [3], and Publius =-=[18]-=-. These systems all require efficient insertion and retrieval of content in a large distributed storage infrastructure, and a scalable indexing mechanism is an essential component. In fact, as we disc...

Citation Context

...‘ª « ¬ ª 13 distributed routing algorithm, both because this does not stress a small set of nodes and it avoids a single point of failure. We hence avoided any form of hierarchical routing algorithms =-=[22, 14, 4]-=-. Perhaps closest in spirit to the CAN routing scheme is the Plaxton algorithm [20]. In Plaxton’s algorithm, every node is assigned a unique n bit label. This n bit label is divided into l levels, wit...

Citation Context

...sights, and seek new insights, to produce even better algorithms. In that spirit we end this thesis with a discussion of issues relevant to routing algorithms and identify some open research questions=-=[38]-=-. Of course, our list of questions is not intended to be exhaustive, merely illustrative. As should be clear by our description, this section is not aboutsnished work, but instead is about a research ...

Citation Context

...ly manner. The JXTA group at SUN Microsystems is building a CAN-based Chat service [2] that uses DHTs to locate resources such as person identiers or group names. The Geographic Hash Table (GHT) [44=-=, 34]-=- is a DHT for large-scale networks of sensors. Although functionally equivalent to DHTs, GHT does not directly adopt the routing algorithms of the DHTs (CAN, Chord, etc.) but instead uses GPSR [22], a...

Citation Context

...ring no communication or cooperation between the nodes being binned). Distributed binning is however, just one of several conceivable approaches to acquiring topological information (see for example, =-=[14, 32, 5, 24-=-]). We use distributed binning because it is simple and lightweight enough to be of practical use in very large scale distributed applications; however any similarly scalable technique that oers reaso...

Citation Context

... truly distributed routing algorithm, both because this does not stress a small set of nodes and it avoids a single point of failure. We hence avoided more traditional hierarchical routing algorithms =-=[2, 24, 16, 6]. Pe-=-rhaps closest in spirit to the CAN routing scheme is the Plaxton algorithm [21]. In Plaxton’s algorithm, every node is assigned a unique n bit label. This � bit label is divided into l levels, wit...

Citation Context

...ghbors. In such 2.3.1 Immediate takeover algorithm When a node fails, one of its neighboring nodes “takes over” the failed node’s now vacant sub-space. Our algorithms draw heavily from the soft-state =-=[21]-=- style recovery algorithms described in [6]. Nodes send periodic update messages to their neighbors. These periodic message updates include the node’s virtual coordinates, and its list of neighbors (b...

Citation Context

...es and so forth [13, 19, 28]. Because of the problems facing the deployment of a network-level multicast service, many recent research proposals have argued for an application-level multicast service =-=[4, 13, 19]-=- as a more tractable alternative to network-level multicast and have described designs and applications for such a service. This chapter looks into the question of how the deployment of a CAN-like dis...

Citation Context

...l services, we describe M-CAN an application-level multicast scheme that builds on CAN. A number of applications such as software distribution, Internet TV, video conferencing and shared white-boards =-=[29, 20, 27, 23]-=- require one-to-many message transmission to enable ecient many-to-many communication. The IP Multicast service [10] was proposed as an extension to the Internet architecture to support ecient multi-p...

Citation Context

...ring no communication or cooperation between the nodes being binned). Distributed binning is however, just one of several conceivable approaches to acquiring topological information (see for example, =-=[14, 32, 5, 24-=-]). We use distributed binning because it is simple and lightweight enough to be of practical use in very large scale distributed applications; however any similarly scalable technique that oers reaso...

Citation Context

... truly distributed routing algorithm, both because this does not stress a small set of nodes and it avoids a single point of failure. We hence avoided more traditional hierarchical routing algorithms =-=[2, 24, 16, 6]. Pe-=-rhaps closest in spirit to the CAN routing scheme is the Plaxton algorithm [21]. In Plaxton’s algorithm, every node is assigned a unique n bit label. This � bit label is divided into l levels, wit...

Citation Context

...l services, we describe M-CAN an application-level multicast scheme that builds on CAN. A number of applications such as software distribution, Internet TV, video conferencing and shared white-boards =-=[29, 20, 27, 23]-=- require one-to-many message transmission to enable ecient many-to-many communication. The IP Multicast service [10] was proposed as an extension to the Internet architecture to support ecient multi-p...

Citation Context

...-service attacks [18, 15, 4, 5]. Additional related problems that are topics for future work include the extension of our CAN algorithms to handle mutable content, and the design of search techniques =-=[13, 9, 25]-=- such as keyword searching built around our CAN indexing mechanism etc. Our interest in exploring the scalability of our design, and the difficulty of conducting truly large scale experiments (hundred...

Citation Context

... not well-suited for our application. Because the reasons involve the details of the algorithm, we discuss them at some length in Appendix C. Algorithms built around the concept of geographic routing =-=[14, 17] are-=- similar to our CAN routing algorithm in that they build around the notion of forwarding messages through a coordinate space. The key difference is that the “space” in their work refers to true ph...

Citation Context

..., is already proving to be a useful substrate for large distributed systems; a number of projects are proposing to build Internet-scale facilities layered above DHTs, including distributedsle systems =-=[11, 25, 9-=-], application-layer multicast [36, 51], event notication services [3, 40], and chat services [2]. With so many applications being developed in so short a time, we expect the DHT functionality to beco...

Citation Context

... configured overlays is common in commercially deployed overlay networks. Finally, a lot of ongoing research effort focuses on understanding network topology and building network measurement services =-=[2, 19, 8]-=-. Some commercial companies [1] even offer services based on technology that help ”map” the Internet. The results of this research and the information provided by various measurement services could be...

Citation Context

...r data set is from measurements taken in April 2001. The NLANR sites are primarily located at universities in North America. The details of the NLANR measurement methodology and sites is described in =-=[17]-=-. 3 These delay assignments are probably quite misleading, since the true Internet latencies are not random; at the very least, they usually obey the triangle inequality. However, we do not yet know h...

Citation Context

...l services, we describe M-CAN an application-level multicast scheme that builds on CAN. A number of applications such as software distribution, Internet TV, video conferencing and shared white-boards =-=[29, 20, 27, 23]-=- require one-to-many message transmission to enable ecient many-to-many communication. The IP Multicast service [10] was proposed as an extension to the Internet architecture to support ecient multi-p...

Citation Context

...are there other properties (such as those described in the following sections) that are made worse in these hybrid routing algorithms? The Viceroy routing algorithm recently proposed by Malkhi et al. =-=[26]-=-, achieves O(log n) pathlengths with O(1) neighbors, thus answering thesrst of the above questions; the latter question { whether some other aspect of routing gets worse { is still, to the best 81 of ...

Citation Context

...uming that all nodes have the same capacity to process messages and then, only later, add on techniques for coping with heterogeneity. 2 However, the heterogeneity observed in current P2P populations =-=[41-=-] is quite extreme, with dierences of several orders of magnitude in bandwidth. One can ask whether the routing algorithms, rather than merely coping with heterogeneity, should instead use it to their...

Citation Context

...-service attacks [18, 15, 4, 5]. Additional related problems that are topics for future work include the extension of our CAN algorithms to handle mutable content, and the design of search techniques =-=[13, 9, 25]-=- such as keyword searching built around our CAN indexing mechanism etc. Our interest in exploring the scalability of our design, and the difficulty of conducting truly large scale experiments (hundred...

Citation Context

... number of projects are proposing to build Internet-scale facilities layered above DHTs, including distributedsle systems [11, 25, 9], application-layer multicast [36, 51], event notication services [=-=3, 40]-=-, and chat services [2]. With so many applications being developed in so short a time, we expect the DHT functionality to become an integral part of the future P2P landscape. 6 The core of these DHT s...

Citation Context

... configured overlays is common in commercially deployed overlay networks. Finally, a lot of ongoing research effort focuses on understanding network topology and building network measurement services =-=[2, 19, 8]-=-. Some commercial companies [1] even offer services based on technology that help ”map” the Internet. The results of this research and the information provided by various measurement services could be...

Citation Context

...cess as well. Users in a Gnutella network self-organize into an application-level mesh on which requests for a file are flooded with a certain scope. Flooding on every request is clearly not scalable =-=[12]-=- and, because the flooding has to be curtailed at some point, may fail to find content that is actually in the system. We started our investigation with the question: could one make a scalable peer-to...

Citation Context

... number of projects are proposing to build Internet-scale facilities layered above DHTs, including distributedsle systems [11, 25, 9], application-layer multicast [36, 51], event notication services [=-=3, 40]-=-, and chat services [2]. With so many applications being developed in so short a time, we expect the DHT functionality to become an integral part of the future P2P landscape. 6 The core of these DHT s...

Citation Context

...de will mostly acquire files with similar keys. The scalability of the above algorithm is yet to be fully studied. Ongoing work at U.C.Berkeley (independent of the OceanStore project described above) =-=[5]-=- looks into developing a peer-to-peer file sharing application using a location algorithm similar to the Plaxton algorithm (although developed independently from the Plaxton work). A novel aspect of t...

Citation Context

...unts for a large fraction of Internet trac, frequently overtaking even the Web in this respect. For example, an October 2001 trace of the trac at multiple border routers within a large ISP's backbone =-=[42]-=-, revealed that a total volume of approximately 1.2 Tera-Bytes/day could be attributed to just three of the more popular P2P systems. Yet another recent report [15] estimates that between 400,000 and ...

Citation Context

... than others thus putting higher load on the nodes hosting those pairs. This is similar to the \hot spot" problem on the Web and can be addressed using caching and replication schemes as discusse=-=d in [35, 45]-=-. 22 7's coordinate neighbor set = { } 1 5 2 3 6 4 (x,y) sample routing path from node 1 to point (x,y) 1's coordinate neighbor set = {2,3,4,5} 7 5 2 3 4 6 1's coordinate neighbor set = {2,3,4,7} 7's ...

Citation Context

...e Plaxton work). A novel aspect of their work is the randomization of path selection for improved robustness. A description and evaluation of these and other file sharing applications can be found at =-=[27]-=-. A key difference between our CAN algorithm and most of these file sharing systems is that under normal operating conditions, content that exists within the CAN can always be located by any other nod...

Citation Context

...orwarded. Unfortunately, currently deployed systems have signicant scaling problems; for example, in Gnutella, searches aresooded with a certain scope;sooding on every request is clearly not scalable[=-=16]-=- and, because thesooding has to be curtailed at some point, may fail tosnd the desiredsle in large systems. FreeNet, another decentralized system, uses a random-walk-based search algorithm that can fa...

Citation Context

...or many years now. Yet, IP multicast deployment has been slowed 57 by dicult issues related to scalable inter-domain routing protocols, charging models, robust congestion control schemes and so forth =-=[13, 19, 28]-=-. Because of the problems facing the deployment of a network-level multicast service, many recent research proposals have argued for an application-level multicast service [4, 13, 19] as a more tracta...

Citation Context

...me point, may fail to find content that is actually in the system. Systems like the Web and the DNS impose strict restrictions on how content may be named which brings its own set of problems+ ! ! 2 =-=[3, 5]-=- while systems like the DNS are heavyweight in terms of configuration, maintainance and updates. In this paper, we define the concept of a ContentAddressable Network; an internet-scale, distributed ha...

Citation Context

...security, accountability etc) tackled by systems like FreeNet, Gnutella and MojoNation, it can serve as a core application building block within all of them.15 4.2.5 Ë Distributed Data Structures In =-=[11]-=-, Gribble et al. implement a distributed hash table designed to run on a cluster of workstations. Their goal is to ease the development of scalable, available services running on a cluster of workstat...

Citation Context

...e difficulty of the problem begins to sink in when we consider some of the large scale indexing systems in existence. Centralised solutions like Napster [17] have scalability problems under high load =-=[18]-=-, suffer from a single point of failure and are expensive. Peer-to-peer systems such as Gnutella [10] locate content by flooding search requests over a self-organised overlay network. While truly dist...

Citation Context

...uters within a large ISP's backbone [42], revealed that a total volume of approximately 1.2 Tera-Bytes/day could be attributed to just three of the more popular P2P systems. Yet another recent report =-=[15]-=- estimates that between 400,000 and 600,000slms are swapped daily over the Internet using systems such as Gnutella, FastTrack, and IRC, with the trading populations of these sytems soaring to 9 millio...

Citation Context

...onality (e.g.,snding correlations over stored data) of traditional database query languages. The PIER project is exploring the design and implementation of complex database query facilities over DHTs =-=[18-=-]. The i3 project [46] proposes the deployment of a general-purpose Internet Indirection Infrastructure { a DHT-based infrastructure that provides a rendezvous-based communication abstraction. The au...

Citation Context

... and more. By every indication, P2P systems have grown into a major Internet application. While the deployment of these systems raises a host of interesting legal, social and network policy questions =-=[43]-=-, from a technical 1 Statistics on the number of software downloads for the varioussle-sharing applications are available at download.com; estimates for the number of simultaneous users are available ...

Citation Context

...ercially deployed overlay networks. Finally, a lot of ongoing research effort focuses on understanding network topology and building network measurement services [2, 19, 8]. Some commercial companies =-=[1]-=- even offer services based on technology that help ”map” the Internet. The results of this research and the information provided by various measurement services could be effectively utilised to incorp...