Results 1 - 10
of
90
The design of the borealis stream processing engine
- In CIDR
, 2005
"... Borealis is a second-generation distributed stream processing engine that is being developed at Brandeis University, Brown University, and MIT. Borealis inherits core stream processing functionality from Aurora [14] and distribution functionality from Medusa [51]. Borealis modifies and extends both ..."
Abstract
-
Cited by 250 (10 self)
- Add to MetaCart
(Show Context)
Borealis is a second-generation distributed stream processing engine that is being developed at Brandeis University, Brown University, and MIT. Borealis inherits core stream processing functionality from Aurora [14] and distribution functionality from Medusa [51]. Borealis modifies and extends both systems in non-trivial and critical ways to provide advanced capabilities that are commonly required by newly-emerging stream processing applications. In this paper, we outline the basic design and functionality of Borealis. Through sample real-world applications, we motivate the need for dynamically revising query results and modifying query specifications. We then describe how Borealis addresses these challenges through an innovative set of features, including revision records, time travel, and control lines. Finally, we present a highly flexible and scalable QoS-based optimization model that operates across server and sensor networks and a new fault-tolerance model with flexible consistency-availability trade-offs.
Multilevel algorithms for partitioning power-law graphs
- IEEE INTERNATIONAL PARALLEL & DISTRIBUTED PROCESSING SYMPOSIUM (IPDPS). IN
, 2006
"... Graph partitioning is an enabling technology for parallel processing as it allows for the effective decomposition of unstructured computations whose data dependencies correspond to a large sparse and irregular graph. Even though the problem of computing high-quality partitionings of graphs arising i ..."
Abstract
-
Cited by 61 (1 self)
- Add to MetaCart
Graph partitioning is an enabling technology for parallel processing as it allows for the effective decomposition of unstructured computations whose data dependencies correspond to a large sparse and irregular graph. Even though the problem of computing high-quality partitionings of graphs arising in scientific computations is to a large extent wellunderstood, this is far from being true for emerging HPC applications whose underlying computation involves graphs whose degree distribution follows a power-law curve. This paper presents new multilevel graph partitioning algorithms that are specifically designed for partitioning such graphs. It presents new clustering-based coarsening schemes that identify and collapse together groups of vertices that are highly connected. An experimental evaluation of these schemes on 10 different graphs show that the proposed algorithms consistently and significantly
Parallel static and dynamic multi-constraint graph partitioning
- CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE
"... ..."
A Social Network Based Patching Scheme for Worm Containment in Cellular Networks
"... Abstract—Recently, cellular phone networks have begun allowing third-party applications to run over certain open-API phone operating systems such as Windows Mobile, Iphone and Google’s Android platform. However, with this increased openness, the fear of rogue programs written to propagate from one p ..."
Abstract
-
Cited by 43 (9 self)
- Add to MetaCart
(Show Context)
Abstract—Recently, cellular phone networks have begun allowing third-party applications to run over certain open-API phone operating systems such as Windows Mobile, Iphone and Google’s Android platform. However, with this increased openness, the fear of rogue programs written to propagate from one phone to another becomes ever more real. This paper proposes a countermechanism to contain the propagation of a mobile worm at the earliest stage by patching an optimal set of selected phones. The counter-mechanism continually extracts a social relationship graph between mobile phones via an analysis of the network traffic. As people are more likely to open and download content that they receive from friends, this social relationship graph is representative of the most likely propagation path of a mobile worm. The counter mechanism partitions the social relationship graph via two different algorithms, balanced and clustered partitioning and selects an optimal set of phones to be patched first as those which have the capability to infect the most number of other phones. The performance of these partitioning algorithms is compared against a benchmark random partitioning scheme. Through extensive trace-driven experiments using real IP packet traces from one of the largest cellular networks in the US, we demonstrate the efficacy of our proposed counter-mechanism in containing a mobile worm. I.
Graph partitioning via adaptive spectral techniques
- Comb. Probab. Comput
"... Abstract. In this paper we study the use of spectral techniques for graph partitioning. Let G = (V,E) be a graph whose vertex set has a “latent ” partition V1,..., Vk. Moreover, consider a “density matrix” E = (Evw)v,w∈V such that for v ∈ Vi and w ∈ Vj the entry Evw is the fraction of all possible V ..."
Abstract
-
Cited by 38 (0 self)
- Add to MetaCart
(Show Context)
Abstract. In this paper we study the use of spectral techniques for graph partitioning. Let G = (V,E) be a graph whose vertex set has a “latent ” partition V1,..., Vk. Moreover, consider a “density matrix” E = (Evw)v,w∈V such that for v ∈ Vi and w ∈ Vj the entry Evw is the fraction of all possible Vi-Vj-edges that are actually present in G. We show that on input (G, k) the partition V1,..., Vk can (almost) be recovered in polynomial time via spectral methods, provided that the following holds: E approximates the adjacency matrix of G in the operator norm, for vertices v ∈ Vi, w ∈ Vj 6 = Vi the corresponding column vectors Ev, Ew are separated, and G is sufficiently “regular ” w.r.t. the matrix E. This result in particular applies to sparse graphs with bounded average degree as n = #V →∞, and it yields interesting consequences on partitioning random graphs.
Engineering a Scalable High Quality Graph Partitioner
- 24TH IEEE INTERNATIONAL PARALLAL AND DISTRIBUTED PROCESSING SYMPOSIUM (IPDPS
, 2010
"... We describe an approach to parallel graph partitioning that scales to hundreds of processors and produces a high solution quality. For example, for many instances from Walshaw’s bench-mark collection we improve the best known partitioning. We use the well known framework of multi-level graph partiti ..."
Abstract
-
Cited by 33 (19 self)
- Add to MetaCart
We describe an approach to parallel graph partitioning that scales to hundreds of processors and produces a high solution quality. For example, for many instances from Walshaw’s bench-mark collection we improve the best known partitioning. We use the well known framework of multi-level graph partitioning. All components are implemented by scalable parallel algo-rithms. Quality improvements compared to previous systems are due to better prioritization of edges to be contracted, better approximation algorithms for identifying matchings, better local search heuristics, and perhaps most notably, a parallelization of the FM local search algorithm that works more locally than previous approaches.
A new diffusion-based multilevel algorithm for computing graph partitions of very high quality
- In Proc. 22nd IPDPS
, 2008
"... Abstract. Graph partitioning requires the division of a graph's vertex set into k equally sized subsets s. t. some objective function is optimized. High-quality partitions are important for many applications, whose objective functions are often NP-hard to optimize. Most state-of-the-art graph p ..."
Abstract
-
Cited by 32 (9 self)
- Add to MetaCart
(Show Context)
Abstract. Graph partitioning requires the division of a graph's vertex set into k equally sized subsets s. t. some objective function is optimized. High-quality partitions are important for many applications, whose objective functions are often NP-hard to optimize. Most state-of-the-art graph partitioning li-braries use a variant of the Kernighan-Lin (KL) heuristic within a multilevel framework. While these libraries are very fast, their solutions do not always meet all user requirements. Moreover, due to its sequential nature, KL is not easy to parallelize. Its use as a load balancer in parallel numerical appli-cations therefore requires complicated adaptations. That is why we developed previously an inherently parallel algorithm, called Bubble-FOS/C (Meyerhenke et al., IPDPS'06), which optimizes partition shapes by a diffusive mechanism. However, it is too slow for practical use, despite its high solution quality. In this paper, besides proving that Bubble-FOS/C converges towards a local optimum of a potential function, we develop a much faster method for the improvement of partitionings. This faster method called TruncCons is based on a different diffusive process, which is restricted to local areas of the graph and also contains a high degree of parallelism. By coupling TruncCons with Bubble-FOS/C in a multilevel framework based on two different hierarchy construction methods, we obtain our new graph
Engineering Multilevel Graph Partitioning Algorithms
"... We present a multi-level graph partitioning algorithm using novel local improvement algorithms and global search strategies transferred from multigrid linear solvers. Local improvement algorithms are based on max-flow min-cut computations and more localized FM searches. By combining these technique ..."
Abstract
-
Cited by 31 (16 self)
- Add to MetaCart
We present a multi-level graph partitioning algorithm using novel local improvement algorithms and global search strategies transferred from multigrid linear solvers. Local improvement algorithms are based on max-flow min-cut computations and more localized FM searches. By combining these techniques, we obtain an algorithm that is fast on the one hand and on the other hand is able to improve the best known partitioning results for many inputs. For example, in Walshaw’s well known benchmark tables we achieve 317 improvements for the tables at 1%, 3 % and 5 % imbalance. Moreover, in 118 out of the 295 remaining cases we have been able to reproduce the best cut in this benchmark.
Multilevel Mesh Partitioning for Heterogeneous Communication Networks
- Future Generation Comput. Syst
, 2001
"... Multilevel algorithms are a successful class of optimisation techniques which address the mesh partitioning problem for distributing unstructured meshes onto parallel computers. They usually combine a graph contraction algorithm together with a local optimisation method which refines the partition a ..."
Abstract
-
Cited by 31 (9 self)
- Add to MetaCart
(Show Context)
Multilevel algorithms are a successful class of optimisation techniques which address the mesh partitioning problem for distributing unstructured meshes onto parallel computers. They usually combine a graph contraction algorithm together with a local optimisation method which refines the partition at each graph level. To date these algorithms have been used almost exclusively to minimise the cut edge weight in the graph with the aim of minimising the parallel communication overhead, but recently there has been a perceived need to take into account the communications network of the parallel machine. For example the increasing use of SMP clusters (systems of multiprocessor compute nodes with very fast intra-node communications but relatively slow inter-node networks) suggest the use of hierarchical network models. Indeed this requirement is exacerbated in the early experiments with meta-computers (multiple supercomputers combined together, in extreme cases over inter-continental networks). In this paper therefore, we modify a multilevel algorithm in order to minimise a cost function based on a model of the communications network. Several network models and variants of the algorithm are tested and we establish that it is possible to successfully guide the optimisation to reflect the chosen architecture. 2001 Elsevier Science B.V. All rights reserved.
Distributed Evolutionary Graph Partitioning
, 2012
"... We present a novel distributed evolutionary algorithm, KaFFPaE, to solve the Graph Partitioning Problem, which makes use of KaFFPa (Karlsruhe Fast Flow Partitioner). The use of our multilevel graph partitioner KaFFPa pro-vides new effective crossover and mutation operators. By combining these with a ..."
Abstract
-
Cited by 27 (13 self)
- Add to MetaCart
We present a novel distributed evolutionary algorithm, KaFFPaE, to solve the Graph Partitioning Problem, which makes use of KaFFPa (Karlsruhe Fast Flow Partitioner). The use of our multilevel graph partitioner KaFFPa pro-vides new effective crossover and mutation operators. By combining these with a scalable communication protocol we obtain a system that is able to improve the best known par-titioning results for many inputs in a very short amount of time. For example, in Walshaw’s well known benchmark ta-bles we are able to improve or recompute 76 % of entries for the tables with 1%, 3 % and 5 % imbalance.
