DMCA
Spidercast: A scalable interest aware overlay for topic-based pub/sub communication (2006)
| Venue: | In Proceedings of the 2007 inaugural international conference on Distributed event-based systems (DEBS 2007 |
| Citations: | 41 - 11 self |
BibTeX
@TECHREPORT{Chockler06spidercast:a,
author = {Gregory Chockler and Roie Melamed and Yoav Tock},
title = {Spidercast: A scalable interest aware overlay for topic-based pub/sub communication},
institution = {In Proceedings of the 2007 inaugural international conference on Distributed event-based systems (DEBS 2007},
year = {2006}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We introduce SpiderCast, a distributed protocol for constructing scalable churn-resistant overlay topologies for supporting decentralized topic-based pub/sub communication. SpiderCast is designed to effectively tread the balance between average overlay degree and communication cost of event dissemination. It employs a novel coverage-optimizing heuristic in which the nodes utilize partial subscription views (provided by a decentralized membership service) to reduce the average node degree while guaranteeing (with high probability) that the events posted on each topic can be routed solely through the nodes interested in this topic (in other words, the overlay is topic-connected). SpiderCast is unique in maintaining an overlay topology that scales well with the average number of topics a node is subscribed to, assuming the subscriptions are correlated insofar as found in most typical workloads. Furthermore, the degree grows logarithmically in the total number of topics, and slowly decreases as the number of nodes increases. We show experimentally that, for many practical workloads, the SpiderCast overlays are both topic-connected and have a low per-topic diameter while requiring each node to maintain a low average number of connections. These properties are satisfied even in very large settings involving up to 10, 000 nodes, 1, 000 topics, and 70 subscriptions per-node, and under high churn rates. In addition, our results demonstrate that, in a large setting, the average node degree in SpiderCast is at least 45 % smaller than in other overlays typically used to support decentralized pub/sub communication (such as e.g., similarity-based, rings-based, and random overlays).
Keyphrases
topic-based pub sub communication scalable interest aware overlay large setting average node degree average overlay degree overlay topology scalable churn-resistant overlay topology high churn rate communication cost event dissemination high probability spidercast overlay many practical workload average number novel coverage-optimizing heuristic low per-topic diameter total number partial subscription view decentralized pub sub communication low average number typical workload node increase decentralized membership service
