Results 11  20
of
50
Mmap: Fast billionscale graph computation on a pc via memory mapping
 in Proceedings of the IEEE International Conference on Big Data. IEEE
, 2014
"... Abstract—Graph computation approaches such as GraphChi and TurboGraph recently demonstrated that a single PC can perform efficient computation on billionnode graphs. To achieve high speed and scalability, they often need sophisticated data structures and memory management strategies. We propose a m ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
Abstract—Graph computation approaches such as GraphChi and TurboGraph recently demonstrated that a single PC can perform efficient computation on billionnode graphs. To achieve high speed and scalability, they often need sophisticated data structures and memory management strategies. We propose a minimalist approach that forgoes such requirements, by leveraging the fundamental memory mapping (MMap) capability found on operating systems. We contribute: (1) a new insight that MMap is a viable technique for creating fast and scalable graph algorithms that surpasses some of the best techniques; (2) the design and implementation of popular graph algorithms for billionscale graphs with little code, thanks to memory mapping; (3) extensive experiments on real graphs, including the 6.6 billion edge YahooWeb graph, and show that this new approach is significantly faster or comparable to the highlyoptimized methods (e.g., 9.5X faster than GraphChi for computing PageRank on 1.47B edge Twitter graph). We believe our work provides a new direction in the design and development of scalable algorithms. Our packaged code is available at
Flashgraph: processing billionnode graphs on an array of commodity ssds
"... Abstract—Graph analysis performs many random reads and writes, thus these workloads are typically performed in memory. Traditionally, analyzing large graphs requires a cluster of machines so the aggregate memory exceeds the size of the graph. We demonstrate that a multicore server can process graph ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract—Graph analysis performs many random reads and writes, thus these workloads are typically performed in memory. Traditionally, analyzing large graphs requires a cluster of machines so the aggregate memory exceeds the size of the graph. We demonstrate that a multicore server can process graphs of billions of vertices and hundreds of billions of edges, utilizing commodity SSDs without much performance loss. We do so by implementing a graphprocessing engine within a userspace SSD file system designed for high IOPS and extreme parallelism. This allows us to localize computation to cached data in a nonuniform memory architecture and hide latency by overlapping computation with I/O. Our semiexternal memory graph engine, called FlashGraph, stores vertex state in memory and adjacency lists on SSDs. FlashGraph exposes a general and flexible programming interface that can express a variety of graph algorithms and their optimizations. FlashGraph in semiexternal memory performs many algorithms up to 20 times faster than PowerGraph, a generalpurpose, inmemory graph engine. Even breadthfirst search, which generates many small random I/Os, runs significantly faster in FlashGraph. I.
PrefEdge: SSD prefetcher for largescale graph traversal
 In Proceedings of the 7th International Systems and Storage Conference, SYSTOR ’14
, 2014
"... Mining large graphs has now become an important aspect of multiple diverse applications and a number of computer systems have been proposed to provide runtime support. Recent interest in this area has led to the construction of single machine graph computation systems that use solid state drives ( ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Mining large graphs has now become an important aspect of multiple diverse applications and a number of computer systems have been proposed to provide runtime support. Recent interest in this area has led to the construction of single machine graph computation systems that use solid state drives (SSDs) to store the graph. This approach reduces the cost and simplifies the implementation of graph algorithms, making computations on large graphs available to the average user. However, SSDs are slower than main memory, and making full use of their bandwidth is crucial for executing graph algorithms in a reasonable amount of time. In this paper, we present PrefEdge, a prefetcher for graph algorithms that parallelises requests to derive maximum throughput from SSDs. PrefEdge combines a judicious distribution of graph state between main memory and SSDs with an innovative readahead algorithm to prefetch needed data in parallel. This is in contrast to existing approaches that depend on multithreading the graph algorithms to saturate available bandwidth. Our experiments on graph algorithms using random access show that PrefEdge not only is capable of maximising the throughput from SSDs but is also able to almost hide the effect of I/O latency. The improvements in runtime for graph algorithms is up to 14 × when compared to a single threaded baseline. When compared to multithreaded implementations, PrefEdge performs up to 80 % faster without the program complexity and the programmer effort needed for multithreaded graph algorithms.
GridGraph: Largescale graph processing on a single machine using 2level hierarchical partitioning
 In Proceedings of the Usenix Annual Technical Conference (2015), USENIX Association
"... (USENIX ATC ’15) is sponsored by USENIX. ..."
(Show Context)
Balanced graph edge partition
 KDD
, 2014
"... Abstract Balanced edge partition has emerged as a new approach to partition an input graph data for the purpose of scaling out parallel computations, which is of interest for several modern data analytics computation platforms, including platforms for iterative computations, machine learning probl ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Abstract Balanced edge partition has emerged as a new approach to partition an input graph data for the purpose of scaling out parallel computations, which is of interest for several modern data analytics computation platforms, including platforms for iterative computations, machine learning problems, and graph databases. This new approach stands in a stark contrast to the traditional approach of balanced vertex partition, where for given number of partitions, the problem is to minimize the number of edges cut subject to balancing the vertex cardinality of partitions. In this paper, we first characterize the expected costs of vertex and edge partitions with and without aggregation of messages, for the commonly deployed policy of placing a vertex or an edge uniformly at random to one of the partitions. We then obtain the first approximation algorithms for the balanced edgepartition problem which for the case of no aggregation matches the best known approximation ratio for the balanced vertexpartition problem, and show that this remains to hold for the case with aggregation up to factor that is equal to the maximum indegree of a vertex. We report results of an extensive empirical evaluation on a set of realworld graphs, which quantifies the benefits of edgevs. vertexpartition, and demonstrates efficiency of natural greedy online assignments for the balanced edgepartition problem with and with no aggregation.
Graphchidb: Simple design for a scalable graph database system  on just a PC
 CoRR
"... We propose a new data structure, Parallel Adjacency Lists (PAL), for efficiently managing graphs with billions of edges on disk. The PAL structure is based on the graph storage model of GraphChi [14], but we extend it to enable online database features such as queries and fast insertions. In additio ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
We propose a new data structure, Parallel Adjacency Lists (PAL), for efficiently managing graphs with billions of edges on disk. The PAL structure is based on the graph storage model of GraphChi [14], but we extend it to enable online database features such as queries and fast insertions. In addition, we extend the model with edge and vertex attributes. Compared to previous data structures, PAL can store graphs more compactly while allowing fast access to both the incoming and the outgoing edges of a vertex, without duplicating data. Based on PAL, we design a graph database management system, GraphChiDB, which can also execute powerful analytical graph computation. We evaluate our design experimentally and demonstrate that GraphChiDB achieves stateoftheart performance on graphs that are much larger than the available memory. GraphChiDB enables anyone with just a laptop or a PC to work with extremely large graphs. 1.
GoFFish: A SubGraph Centric Framework for Largescale Graph Analytics
"... Vertex centric models for large scale graph processing are gaining traction due to their simple distributed programming abstraction. However, pure vertex centric algorithms underperform due to large communication overheads and slow iterative convergence. We introduce GoFFish a scalable subgraph ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
Vertex centric models for large scale graph processing are gaining traction due to their simple distributed programming abstraction. However, pure vertex centric algorithms underperform due to large communication overheads and slow iterative convergence. We introduce GoFFish a scalable subgraph centric framework codesigned with a distributed persistent graph storage for large scale graph analytics on commodity clusters, offering the added natural flexibility of shared memory subgraph computation. We map Connected Components, SSSP and PageRank algorithms to this model and empirically analyze them for several real world graphs, demonstrating orders of magnitude improvements, in some cases, compared to Apache Giraph’s vertex centric framework.
CliqueSquare: efficient Hadoopbased RDF query processing
"... Large volumes of RDF data collections are being created, published and used lately in various contexts, from scientific data to domain ontologies and to open government data, in particular in the context of the Linked Data movement. Managing such large volumes of RDF data is challenging due to the ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Large volumes of RDF data collections are being created, published and used lately in various contexts, from scientific data to domain ontologies and to open government data, in particular in the context of the Linked Data movement. Managing such large volumes of RDF data is challenging due to the sheer size and the heterogeneity. To tackle the size challenge, a single isolated machine is not an efficient solution anymore. The MapReduce paradigm is a promising direction providing scalability and massively parallel processing of largevolume data. We present CliqueSquare, an efficient RDF data management platform based on Hadoop, an open source MapReduce implementation, and its file system, Hadoop Distributed File System (HDFS). CliqueSquare relies on a novel RDF data partitioning scheme enabling queries to be evaluated efficiently, by minimizing both the number of MapReduce jobs and the data transfer between nodes during query execution. We present preliminary experiments comparing our system against HadoopRDF, the stateoftheart Hadoopbased RDF platform. The results demon
Distributed Graph Summarization
"... Graph has been a ubiquitous and essential data representation to model real world objects and their relationships. Today, large amounts of graph data have been generated by various applications. Graph summarization techniques are crucial in uncovering useful insights about the patterns hidden in the ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Graph has been a ubiquitous and essential data representation to model real world objects and their relationships. Today, large amounts of graph data have been generated by various applications. Graph summarization techniques are crucial in uncovering useful insights about the patterns hidden in the underlying data. However, all existing works in graph summarization are singleprocess solutions, and as a result cannot scale to large graphs. In this paper, we introduce three distributed graph summarization algorithms to address this problem. Experimental results show that the proposed algorithms can produce good quality summaries and scale well with increasing data sizes. To the best of our knowledge, this is the first work to study distributed graph summarization methods. 1.
The More the Merrier: Efficient MultiSource Graph Traversal
"... Graph analytics on social networks, Web data, and communication networks has been widely used in a plethora of applications. Many graph analytics algorithms are based on breadthfirst search (BFS) graph traversal, which is not only timeconsuming for large datasets but also involves much redundant ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Graph analytics on social networks, Web data, and communication networks has been widely used in a plethora of applications. Many graph analytics algorithms are based on breadthfirst search (BFS) graph traversal, which is not only timeconsuming for large datasets but also involves much redundant computation when executed multiple times from different start vertices. In this paper, we propose MultiSource BFS (MSBFS), an algorithm that is designed to run multiple concurrent BFSs over the same graph on a single CPU core while scaling up as the number of cores increases. MSBFS leverages the properties of smallworld networks, which apply to many realworld graphs, and enables efficient graph traversal that: (i) shares common computation across concurrent BFSs; (ii) greatly reduces the number of random memory accesses; and (iii) does not incur synchronization costs. We demonstrate how a real graph analytics application—allvertices closeness centrality—can be efficiently solved with MSBFS. Furthermore, we present an extensive experimental evaluation with both synthetic and real datasets, including Twitter and Wikipedia, showing that MSBFS provides almost linear scalability with respect to the number of cores and excellent scalability for increasing graph sizes, outperforming stateoftheart BFS algorithms by more than one order of magnitude when running a large number of BFSs. 1.