Dynamic load distribution in the borealis stream processor (2005)
Cached
Download Links
- [www.cs.brown.edu]
- [www.cs.berkeley.edu]
- [www.cs.brown.edu]
- [now.cs.berkeley.edu]
- DBLP
Other Repositories/Bibliography
| Venue: | In ICDE |
| Citations: | 31 - 4 self |
BibTeX
@INPROCEEDINGS{Xing05dynamicload,
author = {Ying Xing},
title = {Dynamic load distribution in the borealis stream processor},
booktitle = {In ICDE},
year = {2005},
pages = {791--802}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Distributed and parallel computing environments are becoming cheap and commonplace. The availability of large numbers of CPU’s makes it possible to process more data at higher speeds. Stream-processing systems are also becoming more important, as broad classes of applications require results in real-time. Since load can vary in unpredictable ways, exploiting the abundant processor cycles requires effective dynamic load distribution techniques. Although load distribution has been extensively studied for the traditional pull-based systems, it has not yet been fully studied in the context of push-based continuous query processing. In this paper, we present a correlation based load distribution algorithm that aims at avoiding overload and minimizing end-to-end latency by minimizing load variance and maximizing load correlation. While finding the optimal solution for such a problem is NP-hard, our greedy algorithm can find reasonable solutions in polynomial time. We present both a global algorithm for initial load distribution and a pair-wise algorithm for dynamic load migration.
