Results 1  10
of
23
External Memory Data Structures
, 2001
"... In many massive dataset applications the data must be stored in space and query efficient data structures on external storage devices. Often the data needs to be changed dynamically. In this chapter we discuss recent advances in the development of provably worstcase efficient external memory dynami ..."
Abstract

Cited by 76 (32 self)
 Add to MetaCart
In many massive dataset applications the data must be stored in space and query efficient data structures on external storage devices. Often the data needs to be changed dynamically. In this chapter we discuss recent advances in the development of provably worstcase efficient external memory dynamic data structures. We also briefly discuss some of the most popular external data structures used in practice.
Compact representations of simplicial meshes in two and three dimensions
 International Journal of Computational Geometry and Applications
, 2003
"... We describe data structures for representing simplicial meshes compactly while supporting online queries and updates efficiently. Our data structure requires about a factor of five less memory than the most efficient standard data structures for triangular or tetrahedral meshes, while efficiently su ..."
Abstract

Cited by 24 (6 self)
 Add to MetaCart
(Show Context)
We describe data structures for representing simplicial meshes compactly while supporting online queries and updates efficiently. Our data structure requires about a factor of five less memory than the most efficient standard data structures for triangular or tetrahedral meshes, while efficiently supporting traversal among simplices, storing data on simplices, and insertion and deletion of simplices. Our implementation of the data structures uses about 5 bytes/triangle in two dimensions (2D) and 7.5 bytes/tetrahedron in three dimensions (3D). We use the data structures to implement 2D and 3D incremental algorithms for generating a Delaunay mesh. The 3D algorithm can generate 100 Million tetrahedrons with 1 Gbyte of memory, including the space for the coordinates and all data used by the algorithm. The runtime of the algorithm is as fast as Shewchuk’s Pyramid code, the most efficient we know of, and uses a factor of 3.5 less memory overall. 1
TerraStream: From elevation data to watershed hierarchies
 PROC. ACM SYMPOS. ON ADVANCES IN GEOGRAPHIC INFORMATION SYSTEMS
"... We consider the problem of extracting a river network and a watershed hierarchy from a terrain given as a set of irregularly spaced points. We describe TerraStream, a “pipelined” solution that consists of four main stages: construction of a digital elevation model (DEM), hydrological conditioning, e ..."
Abstract

Cited by 20 (12 self)
 Add to MetaCart
(Show Context)
We consider the problem of extracting a river network and a watershed hierarchy from a terrain given as a set of irregularly spaced points. We describe TerraStream, a “pipelined” solution that consists of four main stages: construction of a digital elevation model (DEM), hydrological conditioning, extraction of river networks, and construction of a watershed hierarchy. Our approach has several advantages over existing methods. First, we design and implement the pipeline so each stage is scalable to massive data sets; a single nonscalable stage would create a bottleneck and limit overall scalability. Second, we develop the algorithms in a general framework so that they work for both TIN and grid DEMs. TerraStream is flexible and allows users to choose from various models and parameters, yet our pipeline is designed to reduce (or eliminate) the need for manual intervention between stages. We have implemented TerraStream and present experimental results on real elevation point sets that show that our approach handles massive multigigabyte terrain data sets. For example, we can process a data set containing over 300 million points—over 20GB of raw data—in under 26 hours, where most of the time (76%) is spent in the initial CPUintensive DEM construction stage.
I/OEfficient Construction of Constrained Delaunay Triangulations
"... In this paper, we designed and implemented an I/Oefficient algorithm for constructing constrained Delaunay triangulations. If the number of constraining segments is smaller than the memory size, our algorithm runs in expected O ( N B logM/B N B) I/Os for triangulating N points in the plane, where M ..."
Abstract

Cited by 19 (5 self)
 Add to MetaCart
In this paper, we designed and implemented an I/Oefficient algorithm for constructing constrained Delaunay triangulations. If the number of constraining segments is smaller than the memory size, our algorithm runs in expected O ( N B logM/B N B) I/Os for triangulating N points in the plane, where M is the memory size and B is the disk block size. If there are more constraining segments, the theoretical bound does not hold, but in practice the performance of our algorithm degrades gracefully. Through an extensive set of experiments with both synthetic and real data, we show that our algorithm is significantly faster than existing implementations.
I/Oefficient Point Location using Persistent BTrees
"... We present an external planar point location data structure that is I/Oefficient both in theory and practice. The developed structure uses linear space and answers a query in optimal O(log B N) I/Os, where B is the disk block size. It is based on a persistent Btree, and all previously developed su ..."
Abstract

Cited by 14 (5 self)
 Add to MetaCart
(Show Context)
We present an external planar point location data structure that is I/Oefficient both in theory and practice. The developed structure uses linear space and answers a query in optimal O(log B N) I/Os, where B is the disk block size. It is based on a persistent Btree, and all previously developed such structures assume a total order on the elements in the structure. As a theoretical result of independent interest, we show how to remove this assumption. Most previous theoretical I/Oefficient planer point location structures are relatively complicated and have not been implemented. Based on a bucket approach, Vahrenhold and Hinrichs therefore developed a simple and practical, but theoretically nonoptimal, heuristic structure. We present an extensive experimental evaluation that shows that on a range of realworld Geographic Information Systems (GIS) data, our structure uses fewer I/Os than the structure of Vahrenhold and Hinrichs to answer a query. On a synthetically generated worstcase dataset, our structure uses significantly fewer I/Os.
Approximate Range Searching Using Binary Space Partitions
"... We show how any BSP tree TP for the endpoints of a set of n disjoint segments in the plane can be used to obtain a BSP tree of size O(n · depth(TP)) for the segments themselves, such that the rangesearching efficiency remains almost the same. We apply this technique to obtain a BSP tree of size O(n ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
We show how any BSP tree TP for the endpoints of a set of n disjoint segments in the plane can be used to obtain a BSP tree of size O(n · depth(TP)) for the segments themselves, such that the rangesearching efficiency remains almost the same. We apply this technique to obtain a BSP tree of size O(n log n) such that εapproximate range searching queries with any constantcomplexity convex query range can be answered in O(minε>0{(1/ε) + kε} log n) time, where kε is the number of segments intersecting the εextended range. The same result can be obtained for disjoint constantcomplexity curves, if we allow the BSP to use splitting curves along the given curves. We also describe how to construct a linearsize BSP tree for lowdensity scenes consisting of n objects in R d such that εapproximate range searching with any constantcomplexity convex query range can be done in O(log n + minε>0{(1/ε d−1) + kε}) time. Finally we show how to adapt our structures so that they become I/Oefficient.
Engineering a compact parallel delaunay algorithm in 3d
 In Proceedings of the ACM Symposium on Computational Geometry
, 2006
"... We describe an implementation of a compact parallel algorithm for 3D Delaunay tetrahedralization on a 64processor sharedmemory machine. Our algorithm uses a concurrent version of the BowyerWatson incremental insertion, and a threadsafe spaceefficient structure for representing the mesh. Using t ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
We describe an implementation of a compact parallel algorithm for 3D Delaunay tetrahedralization on a 64processor sharedmemory machine. Our algorithm uses a concurrent version of the BowyerWatson incremental insertion, and a threadsafe spaceefficient structure for representing the mesh. Using the implementation we are able to generate significantly larger Delaunay meshes than have previously been generated—10 billion tetrahedra on a 64 processor SMP using 200GB of RAM. The implementation makes use of a locality based relabeling of the vertices that serves three purposes—it is used as part of the space efficient representation, it improves the memory locality, and it reduces the overhead necessary for locks. The implementation also makes use of a caching technique to avoid excessive decoding of vertex information, a technique for backing out of insertions that collide, and a shared work queue for maintaining points that have yet to be inserted.
Compact Data Structures with Fast Queries
, 2005
"... Many applications dealing with large data structures can benefit from keeping them in compressed form. Compression has many benefits: it can allow a representation to fit in main memory rather than swapping out to disk, and it improves cache performance since it allows more data to fit into the c ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
Many applications dealing with large data structures can benefit from keeping them in compressed form. Compression has many benefits: it can allow a representation to fit in main memory rather than swapping out to disk, and it improves cache performance since it allows more data to fit into the cache. However, a data structure is only useful if it allows the application to perform fast queries (and updates) to the data.
Planar Point Location For Large Data Sets: To Seek Or Not To Seek
, 2000
"... . We present an algorithm for external memory planar point location that is both effective and easy to implement. The base algorithm is an external memory variant of the bucket method by Edahiro, Kokubo and Asano that is combined with Lee and Yang's batched internal memory algorithm for pla ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
. We present an algorithm for external memory planar point location that is both effective and easy to implement. The base algorithm is an external memory variant of the bucket method by Edahiro, Kokubo and Asano that is combined with Lee and Yang's batched internal memory algorithm for planar point location. Although our algorithm is not optimal in terms of its worstcase behavior, weshow its efficiency for both batched and singleshot queries by experiments with realworld data. The experiments show that the algorithm benefits from its mainly sequential disk access pattern and significantly outperforms the fastest algorithm for internal memory. 1 Introduction The wellknown problem of planar point location consists of determining the region of a planar subdivision that contains a given query point. We assume that a planar subdivision is given by N line segments, and that each segment is labeled with the names of the two regions it separates. In this setting, a point locati...