Results 1 - 10
of
35
Constructing Scalable Overlays for Pub-Sub with Many Topics Problems, Algorithms, and Evaluation
"... We investigate the problem of designing a scalable overlay network to support decentralized topic-based pub/sub communication. We introduce a new optimization problem, called Minimum Topic-Connected Overlay (Min-TCO), that captures the tradeoff between the scalability of the overlay (in terms of the ..."
Abstract
-
Cited by 26 (10 self)
- Add to MetaCart
We investigate the problem of designing a scalable overlay network to support decentralized topic-based pub/sub communication. We introduce a new optimization problem, called Minimum Topic-Connected Overlay (Min-TCO), that captures the tradeoff between the scalability of the overlay (in terms of the nodes ’ fanout) and the message forwarding overhead incurred by the communicating parties. Roughly, the Min-TCO problem is as follows: Given a collection of nodes and their subscriptions, connect the nodes using the minimum possible number of edges so that for each topic t, a message published on t could reach all the nodes interested in t by being forwarded by only the nodes interested in t. We show that the decision version of Min-TCO is NPcomplete, and present a polynomial algorithm that approximates
Minimum Maximum Degree Publish-Subscribe Overlay Network Design
"... Abstract—Designing an overlay network for publish/subscribe communication in a system where nodes may subscribe to many different topics of interest is of fundamental importance. For scalability and efficiency, it is important to keep the degree of the nodes in the publish/subscribe system low. It i ..."
Abstract
-
Cited by 11 (1 self)
- Add to MetaCart
(Show Context)
Abstract—Designing an overlay network for publish/subscribe communication in a system where nodes may subscribe to many different topics of interest is of fundamental importance. For scalability and efficiency, it is important to keep the degree of the nodes in the publish/subscribe system low. It is only natural then to formalize the following problem: Given a collection of nodes and their topic subscriptions connect the nodes into a graph which has least possible maximum degree and in such a way that for each topic t, the graph induced by the nodes interested in t is connected. We present the first polynomial time logarithmic approximation algorithm for this problem and prove an almost tight lower bound on the approximation ratio. Our experimental results show that our algorithm drastically improves the maximum degree of publish/subscribe overlay systems. We also propose a variation of the problem by enforcing that each topic-connected overlay network be of constant diameter, while keeping the average degree low. We present a heuristic for this problem which guarantees that each topic-connected overlay network will be of diameter 2 and which aims at keeping the overall average node degree low. Our experimental results validate our algorithm showing that our algorithm is able to achieve very low diameter without increasing the average degree by much. I.
Divide and Conquer Algorithms for Publish/Subscribe Overlay Design
- INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS
, 2010
"... Overlay network design for topic-based publish/subscribe systems is of primary importance because the overlay directly impacts the system’s performance. Determining a topic-connected overlay, in which for every topic the graph induced by nodes interested in the topic is connected, is a fundamental ..."
Abstract
-
Cited by 9 (6 self)
- Add to MetaCart
Overlay network design for topic-based publish/subscribe systems is of primary importance because the overlay directly impacts the system’s performance. Determining a topic-connected overlay, in which for every topic the graph induced by nodes interested in the topic is connected, is a fundamental problem. Existing algorithms for this problem suffer from three key drawbacks: (1) prohibitively high running time cost, (2) requirement of full system knowledge and centralized operation, and (3) constructing overlay from scratch. From a practical point of view, these are all significant limitations. To address these concerns, in this paper, we develop novel algorithms that efficiently solve the problem of dynamically joining two or more topic-connected overlays. Inspired from the divide-and-conquer character of our approach, we derive an algorithm that solves the original problem at a fraction (up to 1.7%) of the running time cost of alternative solutions, but at the expense of an empirically insignificant increase in the average node degree.
Parameterized Maximum and Average Degree Approximation in Topic-based Publish-Subscribe Overlay Network Design
"... Abstract—Publish/subscribe communication systems where nodes subscribe to many different topics of interest are becoming increasingly more common. Designing overlay networks that connect the nodes subscribed to each distinct topic is hence a fundamental problem in these systems. For scalability and ..."
Abstract
-
Cited by 7 (0 self)
- Add to MetaCart
(Show Context)
Abstract—Publish/subscribe communication systems where nodes subscribe to many different topics of interest are becoming increasingly more common. Designing overlay networks that connect the nodes subscribed to each distinct topic is hence a fundamental problem in these systems. For scalability and efficiency, it is important to keep the degree of the nodes in the publish/subscribe system low. Ideally one would like to be able not only to keep the average degree of the nodes low, but also to ensure that all nodes have equally the same degree, giving rise to the following problem: Given a collection of nodes and their topic subscriptions, connect the nodes into a graph with low average and maximum degree such that for each topic t, the graph induced by the nodes interested in t is connected. We present the first polynomial time parameterized sublinear approximation algorithm for this problem. We also propose two heuristics for constructing topicconnected networks with low average degree and constant diameter and validate our results through simulations. In fact, the results in this section are a refinement of the preliminary results by Onus and Richa in INFOCOM’09. I.
A generalized algorithm for publish/subscribe overlay design and its fast implementation
, 2012
"... It is a challenging and fundamental problem to construct the underlying overlay network to support efficient and scalable information distribution in topic-based publish/subscribe systems. Existing overlay design algorithms aim to minimize the node fan-out while building topic-connected overlays, ..."
Abstract
-
Cited by 4 (4 self)
- Add to MetaCart
It is a challenging and fundamental problem to construct the underlying overlay network to support efficient and scalable information distribution in topic-based publish/subscribe systems. Existing overlay design algorithms aim to minimize the node fan-out while building topic-connected overlays, in which all nodes interested in the same topic are organized in a directly connected dissemination sub-overlay. However, most state-of-the-art algorithms suffer from high computational complexity, such as O(|V | 4 |T |), where V is the node set and T is the topic set. We devise a general indexing data structure that provides a significantly faster implementation, with O(|V | 2 |T |) running time, for different state-of-the-art algorithms. The generality of the indexing data structure is due to the fact that it enables edge lookup by both node degree and edge contribution, a central metric in all existing algorithms. When tested on typical pub/sub workloads, the speedup observed was by a factor of over 1 000, thereby rendering the algorithms more suitable for practical use. For example, under a typically Zipf distributed pub/sub workload, with 1 000 nodes and 100 topics, our new implementation completes in 3.823 seconds, while the previous alternative takes over 555 minutes.
Providing probabilistic latency bounds for dynamic publish/subscribe systems
- In Proceedings of the 16th ITG/GI Conference on Kommunikation in Verteilten Systemen (KiVS
, 2009
"... Abstract. In the context of large decentralized many-to-many communication systems it is impractical to provide realistic and hard bounds for certain QoS metrics including latency bounds. Nevertheless, many applications can yield bet-ter performance if such bounds hold with a given probability. In t ..."
Abstract
-
Cited by 4 (2 self)
- Add to MetaCart
(Show Context)
Abstract. In the context of large decentralized many-to-many communication systems it is impractical to provide realistic and hard bounds for certain QoS metrics including latency bounds. Nevertheless, many applications can yield bet-ter performance if such bounds hold with a given probability. In this paper we show how probabilistic latency bounds can be applied in the context of pub-lish/subscribe. We present an algorithm for maintaining individual probabilis-tic latency bounds in a highly dynamic environment for a large number of sub-scribers. The algorithm consists of an adaptive dissemination algorithm as well as a cluster partitioning scheme. Together they ensure i) adaptation to the individual latency requirements of subscribers under dynamically changing system proper-ties, and ii) scalability by determining appropriate clusters according to available publishers in the system. 1
Generating Wide-Area Content-Based publish/subscribe workloads
"... Content-based publish/subscribe systems allow events to be selectively and aperiodically pushed to subscribers according to their interests, expressed as predicates in a high-dimensional event space. Making such systems scalable in a wide-area network requires considering multiple factors, e.g., dis ..."
Abstract
-
Cited by 4 (1 self)
- Add to MetaCart
Content-based publish/subscribe systems allow events to be selectively and aperiodically pushed to subscribers according to their interests, expressed as predicates in a high-dimensional event space. Making such systems scalable in a wide-area network requires considering multiple factors, e.g., distribution of events, similarity of subscriber interests in the event space, and proximity of subscriber locations in the network. A major obstacle for this research is the lack of publicly available, realistic workloads, because of concerns of privacy and commercial interests in releasing user information. This paper describes a workload generator for wide-area content-based publish/subscribe systems, which extrapolates the limited amount of various statistics available to public, and generates a workload consistent with these statistics. The generator allows users to deviate the workload from the given statistics in meaningful ways, such as lowering the variance. Our hope is that this generator will help publish/subscribe researchers evaluate their research.
Reconfiguration Primitives for Self-adapting Overlays in Distributed Publish-Subscribe Systems
- In SASO
, 2012
"... Abstract—Most distributed applications involve some form of event-based interaction, often implemented using a publish-subscribe (pub-sub) infrastructure. To improve scalability, the acts of matching events against subscriptions and delivery them are performed collaboratively by a set of brokers con ..."
Abstract
-
Cited by 3 (0 self)
- Add to MetaCart
(Show Context)
Abstract—Most distributed applications involve some form of event-based interaction, often implemented using a publish-subscribe (pub-sub) infrastructure. To improve scalability, the acts of matching events against subscriptions and delivery them are performed collaboratively by a set of brokers connected into an overlay network. Recent research has proposed several approaches to support the self-adaptation of such overlay network to adapt it to changes in application traffic. However these approaches focus on the monitor, analyze, plan parts of the self-adaptation loop, without considering the issues that arise in the execution part. This paper proposes a set of primitives that fills the gap in the execution phase. Compared to existing work, our approach: (i) is transparent w.r.t. the routing policies of the middleware, (ii) preserves existing properties and guarantees of the middleware, such as no duplication of events, causal ordering, and minimal delays for the events delivered during a reconfiguration. We discuss the correctness of our primitives and implement them in a simulated environment to measure their cost in terms of network overhead. Keywords-Pub-sub systems; topology management; self-adaptation actions; I.
On interest locality in content-based routing for largescale manets
- In IEEE 6th International Conference on Mobile Adhoc and Sensor Systems (MASS
, 2009
"... Abstract—To disseminate content with content-based routing (CBR), the routing paths of subscription and publication cannot be determined a priori and have to be computed hop-by-hop, which brings in scalability and robustness challenges in large scale mobile ad hoc networks (MANETs). In this paper, w ..."
Abstract
-
Cited by 2 (2 self)
- Add to MetaCart
(Show Context)
Abstract—To disseminate content with content-based routing (CBR), the routing paths of subscription and publication cannot be determined a priori and have to be computed hop-by-hop, which brings in scalability and robustness challenges in large scale mobile ad hoc networks (MANETs). In this paper, we propose a novel two-tier content-based routing protocol called CLONE (Community and Location aware cONtEnt based routing). In CLONE, we map the human community structure of social networks to MANETs. The whole network can be selforganized into communities based on the interest locality, so that most subscriptions inside a community can be served in an intracommunity fashion, reducing the communication overhead and the response delay. Community construction is self-organized and completely distributed. Analytical and simulation results demonstrate the effectiveness of CLONE in large-scale MANETs. I.
