Content-based publish/subscribe systems allow events to be selectively and aperiodically pushed to subscribers according to their interests, expressed as predicates in a high-dimensional event space. Making such systems scalable in a wide-area network requires considering multiple factors, e.g., distribution of events, similarity of subscriber interests in the event space, and proximity of subscriber locations in the network. A major obstacle for this research is the lack of publicly available, realistic workloads, because of concerns of privacy and commercial interests in releasing user information. This paper describes a workload generator for wide-area content-based publish/subscribe systems, which extrapolates the limited amount of various statistics available to public, and generates a workload consistent with these statistics. The generator allows users to deviate the workload from the given statistics in meaningful ways, such as lowering the variance. Our hope is that this generator will help publish/subscribe researchers evaluate their research.

Abstract not found

We study the problem of assigning subscribers to brokers in a wide-area content-based publish/subscribe system. A good assignment should consider both subscriber interests in the event space and subscriber locations in the network space, and balance multiple performance criteria including bandwidth, delay, and load balance. The resulting optimization problem is NP-complete, so systems have turned to heuristics and/or simpler algorithms that ignore some performance criteria. Evaluating these approaches has been challenging because optimal solutions remain elusive for realistic problem sizes. To enable proper evaluation, we develop a Monte Carlo approximation algorithm with good theoretical properties and robustness to workload variations. To make it computationally feasible, we combine the ideas of linear programming, randomized rounding, coreset, and iterative reweighted sampling. We demonstrate how to use this algorithm as a yardstick to evaluate other algorithms, and why it is better than other choices of yardsticks. With its help, we show that a simple greedy algorithm works well for a number of workloads, including one generated from publicly available statistics on Google Groups. We hope that our algorithms are not only useful in their own right, but our principled approach toward evaluation will also be useful in future evaluation of solutions to similar problems in content-based publish/subscribe.

Resource management in large scale distributed systems

Abstract—A publisher subscriber system is an event notification service where events generated by the publishers are routed to subscribers with matching subscriptions. Our work on publisher-subscriber systems in peer-to-peer (p2p) networks is motivated by the question: ”What if the nodes in the network do not have the bandwidth to route all the events generated? ” In such a case nodes may not receive all notifications because of lack of bandwidth, and even those with high bandwidth may starve for notifications if its neighbours do not have the bandwidth to send event notifications. In this paper, we consider Sub-2-Sub architecture for content based publisher subscriber systems and show with the help of simulations that the event dissemination rate will actually be much slower than expected when the publishing rate is very high. We propose a topology design and event dissemination mechanism which removes nodes which become bottleneck in dissemination. Side by side, it allows a node to receive more events if it has more bandwidth. Our system is built on top of Sub-2-Sub publisher subscriber system, and provides much improved performance over it in terms of percent of event received by nodes, and ratio of uploads to downloads (which shows the removal of bottleneck nodes in dissemination). I.