Results 1 - 10
of
42
Gossiping in Distributed Systems
"... Gossip-based algorithms were first introduced for reliably disseminating data in large-scale distributed systems. However, their simplicity, robustness, and flexibility make them attractive for more than just pure data dissemination alone. In particular, gossiping has been applied to data aggregatio ..."
Abstract
-
Cited by 50 (0 self)
- Add to MetaCart
(Show Context)
Gossip-based algorithms were first introduced for reliably disseminating data in large-scale distributed systems. However, their simplicity, robustness, and flexibility make them attractive for more than just pure data dissemination alone. In particular, gossiping has been applied to data aggregation, overlay maintenance, and resource allocation. Gossiping applications more or less fit the same framework, with often subtle differences in algorithmic details determining divergent emergent behavior. This divergence is often difficult to understand, as formal models have yet to be developed that can capture the full design space of gossiping solutions. In this paper, we present a brief introduction to the field of gossiping in distributed systems, by providing a simple framework and using that framework to describe solutions for various application domains.
Distributed slicing in dynamic systems
, 2006
"... Peer to peer (P2P) systems are moving from application specific architectures to a generic service oriented design philosophy. This raises interesting problems in connection with providing useful P2P middleware services capable of dealing with resource assignment and management in a large-scale, het ..."
Abstract
-
Cited by 30 (11 self)
- Add to MetaCart
(Show Context)
Peer to peer (P2P) systems are moving from application specific architectures to a generic service oriented design philosophy. This raises interesting problems in connection with providing useful P2P middleware services capable of dealing with resource assignment and management in a large-scale, heterogeneous and unreliable environment. The slicing service, has been proposed to allow for an automatic partitioning of P2P networks into groups (slices) that represent a controllable amount of some resource and that are also relatively homogeneous with respect to that resource. In this paper we propose two gossip-based algorithms to solve the distributed slicing problem. The first algorithm speeds up an existing algorithm sorting a set of uniform random numbers. The second algorithm statistically approximates the rank of nodes in the ordering. The scalability, efficiency and resilience to dynamics of both algorithms rely on their gossip-based models. These algorithms are proved viable theoretically and experimentally.
Compositional gossip: a conceptual architecture for designing gossip-based applications
- ACM SIGOPS Operating Systems Review
, 2007
"... Most proposed gossip-based systems use an ad-hoc design. We observe a low degree of reutilization among this proposals. We present how this limits both the systematic development of gossip-based applications and the number of applications that can benefit from gossip-based construction. We posit tha ..."
Abstract
-
Cited by 14 (3 self)
- Add to MetaCart
(Show Context)
Most proposed gossip-based systems use an ad-hoc design. We observe a low degree of reutilization among this proposals. We present how this limits both the systematic development of gossip-based applications and the number of applications that can benefit from gossip-based construction. We posit that these reinvent-the-wheel approaches poses a significant barrier to the spread and usability of gossip protocols. This paper advocates a conceptual design framework based upon aggregating basic and predefined building blocks (B 2). We show how to compose building blocks within our framework to construct more complex blocks to be used in gossipbased applications. The concept is further depicted with two gossip-based applications described using our building blocks.
Adam2: Reliable distribution estimation in decentralised environments
- in Proceedings of the 30th IEEE International Conference on Distributed Computing Systems (ICDCS
, 2010
"... Abstract—To enable decentralised actions in very large distributed systems, it is often important to provide the nodes with global knowledge about the values of attributes across all nodes. This paper shows how, given an attribute whose values are distributed across a large decentralised system, eac ..."
Abstract
-
Cited by 14 (3 self)
- Add to MetaCart
Abstract—To enable decentralised actions in very large distributed systems, it is often important to provide the nodes with global knowledge about the values of attributes across all nodes. This paper shows how, given an attribute whose values are distributed across a large decentralised system, each node can efficiently estimate the statistical distribution of these values. Simulations using heavily skewed real-world node attribute distributions show that our estimation methods outperform the state-of-the-art heuristics by an order of magnitude with an average error of 0.05 % and a maximum error of 2%. To obtain this accuracy, each node sends on average just 120 kB of data independent of the system size. Our algorithms also achieve this accuracy in the presence of heavy churn of system membership. Furthermore, our algorithm enables self-tuning by continuously estimating the accuracy of its own distribution approximation. I.
Autonomous resource selection for decentralized utility computing.
- In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS). DAS-3. http://www.cs.vu.nl/das3/.
, 2009
"... Abstract Many large-scale utility computing infrastructures comprise heterogeneous hardware and software resources. This raises the need for scalable resource selection services, which identify resources that match application requirements, and can potentially be assigned to these applications. We ..."
Abstract
-
Cited by 12 (3 self)
- Add to MetaCart
(Show Context)
Abstract Many large-scale utility computing infrastructures comprise heterogeneous hardware and software resources. This raises the need for scalable resource selection services, which identify resources that match application requirements, and can potentially be assigned to these applications. We present a fully decentralized resource selection algorithm by which resources autonomously select themselves when their attributes match a query. An application specifies what it expects from a resource by means of a conjunction of (attribute,value-range) pairs, which are matched against the attribute values of resources. We show that our solution scales in the number of resources as well as in the number of attributes, while being relatively insensitive to churn and other membership changes such as node failures.
Design and Implementation of a P2P Cloud System
"... Cloud Computing has gained popularity in both research and industrial communities. Cloud users can acquire computing resources on a need basis, achieving on demand scalability; Cloud providers can maximize resource utilizations of datacenters, increasing their return on investments. While Cloud syst ..."
Abstract
-
Cited by 9 (0 self)
- Add to MetaCart
(Show Context)
Cloud Computing has gained popularity in both research and industrial communities. Cloud users can acquire computing resources on a need basis, achieving on demand scalability; Cloud providers can maximize resource utilizations of datacenters, increasing their return on investments. While Cloud systems are usually hosted in large datacenters and are centrally managed, other types of Cloud architectures can be imagined. In this paper we describe the design and prototype implementation of a fully decentralized, P2P Cloud. A P2P Cloud allows organizations or even individual to build a computing infrastructure out of existing resources, which can be easily allocated among different tasks. We focus on the problem of maintaining a coherent structure over a set of unreliable computing resources. We show that gossip-based protocols can be used to maintain an overlay network on top of the computing nodes, and to partition the set of resources into multiple slices in such a way that the failure of individual nodes do not compromise the overall structure. Resource partitioning is one of the most important features of a Cloud, and therefore must be supported efficiently and reliably on any Cloud architecture. We describe a prototype Java implementation that is being developed to demonstrate the effectiveness of the proposed approach.
Gossip-based Networking for Internet-Scale Distributed Systems
"... Abstract. In the era of Internet-scale applications, an increasing number of services are distributed over pools of thousands to millions of networked computers. Along with the obvious advantages in performance and capacity, such a massive scale comes also with challenges. Continuous changes in the ..."
Abstract
-
Cited by 9 (1 self)
- Add to MetaCart
(Show Context)
Abstract. In the era of Internet-scale applications, an increasing number of services are distributed over pools of thousands to millions of networked computers. Along with the obvious advantages in performance and capacity, such a massive scale comes also with challenges. Continuous changes in the system become the norm rather than the exception, either because of inevitable hardware failures or merely due to standard maintenance and upgrading procedures. Rather than trying to impose rigid control on the massive pools of resources, we should equip Internet-scale applications with enough flexibility to work around inevitable faults. In that front, gossiping protocols have emerged as a promising component due to their highly desirable properties: self-healing, self-organizing, symmetric, immensely scalable, and simple. Through visiting a representative set of fundamental gossiping protocols, this paper provides insight on the principles that govern their behavior. By focusing on the rationale and incentives behind gossiping protocols, we introduce the reader to the alternative way of managing massive scale systems through gossiping, and we intrigue her or his interest to delve deeper into the subject by providing an extensive list of pointers. 1
Managing clouds: a case for a fresh look at large unreliable dynamic networks
- SIGOPS Oper. Syst. Rev
"... Peer-to-peer (P2P) protocols have proven efficient to provide scalable support to many large-scale distributed applications, successfully coping with unreliability and dynamics. However, to exploit them in a wider range of environments, such as very large-scale networks of smartphones or set-top box ..."
Abstract
-
Cited by 8 (2 self)
- Add to MetaCart
(Show Context)
Peer-to-peer (P2P) protocols have proven efficient to provide scalable support to many large-scale distributed applications, successfully coping with unreliability and dynamics. However, to exploit them in a wider range of environments, such as very large-scale networks of smartphones or set-top boxes, it is imperative to make P2P protocols manageable: we need to be able to start, bootstrap and stop protocols, and assign resources dynamically. In this paper we present a general-purpose framework aimed to support several fully distributed applications running independently over a very large scale and dynamic pool of resources. We call this resource pool a cloud. The basic idea of the framework is a declarative application suit description, that describes what applications should be running on what resources, and a middleware that makes sure the currently available and dynamic cloud self-organizes into the configuration represented by the description, creating the subclouds that are assigned to applications. The middleware also provides additional functionality, such as bootstrapping overlay networks, to support the applications. Our preliminary ideas on the implementation rely on various gossip-based protocols, that are applied to form the subclouds and to implement bootstrapping, monitoring and control services. Most of all, this position paper sets an exciting research agenda to fully exploit the possibilities offered by very large scale dynamic networks. 1.
Slicing Distributed Systems
"... Abstract—Peer-to-peer (P2P) architectures are popular for tasks such as collaborative download, VoIP telephony, and backup. To maximize performance in the face of widely variable storage capacities and bandwidths, such systems typically need to shift work from poor nodes to richer ones. Similar requ ..."
Abstract
-
Cited by 8 (0 self)
- Add to MetaCart
(Show Context)
Abstract—Peer-to-peer (P2P) architectures are popular for tasks such as collaborative download, VoIP telephony, and backup. To maximize performance in the face of widely variable storage capacities and bandwidths, such systems typically need to shift work from poor nodes to richer ones. Similar requirements are seen in today’s large data centers, where machines may have widely variable configurations, loads and performance. In this paper, we consider the slicing problem, which involves partitioning the participating nodes into k subsets using a one-dimensional attribute, and updating the partition as the set of nodes and their associated attributes change. The mechanism thus facilitates the development of adaptive systems. We begin by motivating this problem statement and reviewing prior work. Existing algorithms are shown to have problems with convergence, manifesting as inaccurate slice assignments, and to adapt slowly as conditions change. Our protocol, Sliver, has provably rapid convergence, is robust under stress, and is simple to implement. We present both theoretical and experimental evaluations of the protocol.
Intelligent gossip
- in: International Symposium on Intelligent Distributed Computing
, 2008
"... Summary. The gossip paradigm made its first appearance in distributed systems in 1987, when it was applied to disseminate updates in replicated databases. Two decades later, gossip-based protocols have gone far beyond dissemination, solving a large and diverse collection of problems. We believe that ..."
Abstract
-
Cited by 6 (0 self)
- Add to MetaCart
(Show Context)
Summary. The gossip paradigm made its first appearance in distributed systems in 1987, when it was applied to disseminate updates in replicated databases. Two decades later, gossip-based protocols have gone far beyond dissemination, solving a large and diverse collection of problems. We believe that the story is not over: while gossip is not the panacea for distributed systems, there are still virgin research areas where it could be profitably exploited. In this paper, we briefly discuss a gossipbased “construction set ” for distributed systems and we illustrate how intelligent distributed computing could benefit by the application of its building blocks. Simple examples are provided to back up our claim. 1 Introduction to Gossip Since the seminal paper of Demers et al. [3], the idea of epidemiological (or gossip) algorithms has gained considerable popularity within the distributed systems and algorithms communities. In a recent workshop on the future of gossip (summarized on a special issue
