L. Arge. The buffer tree: A new technique for optimal I/O algorithms. In Proc. 4th WADS, volume 955 of LNCS, pages 334--45. Springer-Verlag, 1995.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....and P ( Delta) can be implemented as a simple queue. THEOREM 3.1. DFS and BFS numbers can be assigned to a directed graph G in O( V E B ) log 2 V B sort(E) I Os. PROOF: We arrange the arcs of G into adjacency and reverse adjacency lists in O(sort(E) I Os. We use Arge s buffer trees [2] for the priority queues. There are E inserts and deletes and 2V delete mins overall, for an I O complexity of O(sort(E) for priority queue operations. There are E inserts and 2V extracts in the BRT, for an overall I O complexity of O( V E B ) log 2 V B ) for BRT operations. 2 3.1 ....

L. Arge. The buffer tree: A new technique for optimal I/O algorithms. In Proc. 4th WADS, volume 955 of LNCS, pages 334--45. Springer-Verlag, 1995.

....We now examine existing rectangle join algorithms. The algorithms of [21, 23, 22, 26, 6, 9, 13, 25, 18, 27, 14] cannot guarantee optimal worst case I O complexity. In the worst cases, the I O complexity could be quadratic even when the number of rectangle intersections is small. The algorithms in [1] and [29] have desirable I O complexity. But both algorithms require index structures on the x projections of rectangles to be constructed before processing the join. This makes pipelining impossible and causes additional I Os and storage space. The I O efficient algorithm in [2] does not use any ....

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data structures, 1995.

....B ) I Os for every Bth Insert and every Bth DeleteMin operation in the worst case. The above structures assume that the internal memory can only fit O(B) elements, i.e. a constant number of blocks. Even faster solutions are possible if the whole capacity of the internal memory is utilized. Arge [5, 6] introduced an (a,b) tree structure that can be used to carry out any sequence of S Insert and DeleteMin operations with O( S B log M=B S M ) I Os. Fadel et al. 13] gave a heap structure with a similar I O performance but their bound depends on the size profile, not on S. Their heap structure ....

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proceedings of the 4th Workshop on Algorithms and Data Structures, Lecture Notes in Computer Science 955, Springer, Berlin/Heidelberg, pages 334--345, 1995.

....B being at least polylogarithmic in the problem size N is reasonable, just as the transdichotomous model [14] specifies that the number of bits in a machine word is logarithmic in the problem size. Further 1 As with B trees, our goal is to optimize query time, not update time as in buffer trees [7]. more, we have the most to gain from locality of reference when the block size is large, so our data structures are optimized for these cases. As memory hierarchies get deeper and the lowest level becomes farther from the CPU, our data structures become more important. We improve the insertion ....

L. Arge. The buffer tree: A new technique for optimal I/Oalgorithms. In Proc. 4th Internat. Workshop on Algorithms and Data Structures, LNCS 955, pp. 334--345, Kingston, Canada, Aug. 1995.

....pushed; and P ( can be implemented as a simple queue. THEOREM 3.1. DFS and BFS numbers can be assigned to a directed graph G in O( V E B ) log 2 V B sort(E) I Os. PROOF: We arrange the arcs of G into adjacency and reverse adjacency lists in O(sort(E) I Os. We use Arge s buffer trees [2] for the priority queues. There are E inserts and deletes and 2V delete mins overall, for an I O complexity of O(sort(E) for priority queue operations. There are E inserts and 2V extracts in the BRT, for an overall I O complexity of O( V E B ) log 2 V B ) for BRT operations. 2 3.1 ....

L. Arge. The buffer tree: A new technique for optimal I/O algorithms. In Proc. 4th WADS, volume 955 of LNCS, pages 334--45. Springer-Verlag, 1995.

.... external memory algorithms is the I O model of [9, 10] In this model, B is used as the amount of block transfer between internal and external memory, M specifies the size of the internal memory, Previous Work In external memory, there are also many priority queue implementations like buffer trees[5], M=B ary heaps[7, 6] All these structures are based on trees and achieve amortized 1=B log M=B N=B I O per operation in a total sequence of N operations. Unfortunately many of these structures are too complicated. Buffer trees for example use internal (a; b) tree rebalancing strategies. 6] s ....

....every update operation will boil down to merging and sorting steps. Consequently, our data structure can be naturally extended to work in the parallel disk model without any loss of efficiency, thus achieving the first optimal result in this setting. In fact, all the previously known solutions [5, 7, 6] adopted the disk striping technique that introduces a slowdown factor of (log M=B) log M=DB) Let us introduce the following notation: N is the total number of operations executed on the heap, B is the disk page size and M is the internal memory size. We define L = log cM=B N=B. Throughout this ....

L. Arge, `The Buffer Tree: A new technique for optimal I/O-algorithms', In Proc. Workshop on Algorithms and Data Struc., 1995.

....of the PRAM algorithm together with a general simulation tool. In contrast, a major goal of our work is to provide simple, direct, and implementable algorithms. 2.2 Buffering Data Structures Another recent approach is based on external variants of classical internal data structures. Arge [3] introduces buffer trees, which support sequences of insert, delete, and deletemin operations on N elements in O( 1 B log b N B ) amortized I Os each. Kumar and Schwabe [15] introduce a variant of the buffer tree, achieving the same heap bounds as Arge. These bounds are optimal, since the ....

....but that are simpler to describe and implement. Our algorithms are conducive to standard checkpointing and programming language optimization tools. An interesting open question is to devise incremental and dynamic algorithms for external graph problems. The data structural approach of Arge [3] and Kumar and Schwabe [15] holds promise for this area. Designing external graph algorithms that exploit parallel disks also remains open. Treating P disks as one with 12 a block size of PB extends standard algorithms only when P = O( M=B) ff ) for 0 ff 1. In this case, the log M=B terms ....

L. Arge. The buffer tree: A new technique for optimal I/O algorithms. In Proc. 4th WADS, volume 955 of LNCS, pages 334--45. Springer-Verlag, 1995.

....case. The main idea underlying these structures is to construct high degree trees instead of binary trees. For example, variants of B trees are used to answer 1 dimensional range searching queries [35, 92] A number of additional tricks are developed to optimize the size and the query time. See [21, 22, 210] for I O efficient data structures that have been used for answering range searching and related queries. Table 2 summarizes the known results on secondary memory structures for orthogonal range searching. The data structure by Subramanian and Ramaswamy [240] for 3 sided queries supports ....

L. Arge, The Buffer Tree: A new technique for optimal I/O-algorithms, Proc. 4th Workshop Algorithms Data Struct., number 955 in Lecture Notes Comput. Sci., 1995, pp. 334--345.

....If nbytes is positive, then both the mtime and ctime of the file are updated. 4 Applications of zero( 4.1 I O Efficient Computation In recent years, I O efficient algorithms for a wide variety of problems have been developed. These include algorithms from the domains of sorting [AV88, Arg94, NV90, NV93, VS94] permuting [CSW94, Cor93, Cor92] computational geometry [GTVV93] line segment intersection [AVV95] graph algorithms [CGG 95] and scientific computation [VV95] The TPIE library [Ven95, Ven94] supports efficient implementation of many of these algorithms. A common theme ....

Lars Arge. The buffer tree: A new technique for optimal I/O-algorithms. Technical Report RS-94-16, BRICS, Univ. of Aarhus, Denmark, 1994.

....observed by several researchers that d ary heaps perform better than the classical binary heaps on multi level memory systems [16, 14] Consequently, a variety of external PQs already known in the literature follow this design paradigm by using a multi way tree as a basic structure. Buffer trees [2, 11] and M=B ary heaps [9, 13] achieve optimal O( 1=B) log M=B N=B) amortized I Os per operation, in a sequence of total N operations. Unfortunately, most of these data structures are quite complex to be implemented (the simplest proposal is in [9] and the constants hidden in the space and I O ....

....the practical setting and thus deserves a careful experimental analysis to validate its conjectured superiority over the other PQs. In the second part of the paper, we will perform an extensive set of experiments comparing the implementation of four external memory PQs: one based on Buffer Trees [2], another based on B Trees [3] and our two new proposals R Heaps and Array Heaps. Additionally, we will compare these PQs against four internal memory priority queues: Fibonacci heaps, k ary heaps, pairing heaps and internal radix heaps. Our experimental framework includes some simple tests, ....

[Article contains additional citation context not shown here]

L. Arge, `The Buffer Tree: A new technique for optimal I/O-algorithms', Workshop on Algorithms and Data Structures, LNCS 955, 334-345, 1995.

....Although this cost model does not account for the variation in disk access times caused by head movement and rotational latency, programmers often have no control over these factors. The number of disk accesses, however, can be minimized by carefully designed algorithms such as those in [AP94, Arg95, CGG 95, Cor92, Cor93, CSW94, GTVV93, NV93, VS94] and this paper. There are two restrictions implied by the Vitter Shriver model. In order for the memory to accomodate the records transferred in a parallel I O operation to all D disks, we require that BD M . Also, we assume that M N , ....

....N Gamma 1g 1 1 f0; 1; N Gamma 1g. The first term comes into play when the block size B is small, and the second term is the sorting bound Theta i N BD lg(N=B) lg(M=B) j , which was shown by Vitter and Shriver for randomized sorting and by Nodine and Vitter [NV93] and by Arge [Arg95] for deterministic sorting. These bounds are asymptotically tight, for they match the lower bounds proven earlier by Aggarwal and Vitter [AV88] using a model with one disk and D independent read write heads, which is at least as powerful as the Vitter Shriver model. Specific classes of ....

Lars Arge. The buffer tree: A new technique for optimal I/O-algorithms. In 4th International Workshop on Algorithms and Data Structures (Proceedings), Lecture Notes in Computer Science, number 955, pages 334--345. Springer-Verlag, August 1995.

....algorithms developed using the methods presented in this thesis. Motivated by the goal of constructing I O efficient versions of commonly used in 2 M is the total size of the internal memories and P 0 is the number of processors CHAPTER 1. INTRODUCTION 11 ternal memory data structures, Arge [8, 9] proposed the data structuring paradigm, and in particular the buffer tree. A buffer tree is an external memory search tree, based on the (a; b) tree [62] which is a generalization of the B tree [27] It allows several update operations, such as insert, delete, search, deletemin, and it ....

....[27] It allows several update operations, such as insert, delete, search, deletemin, and it enables the transformation of a class of internal memory algorithms to external memory algorithms by exchanging the data structures used. A large number of external memory algorithms have been proposed [8, 9] using the buffer tree data structure, including sorting, priority queues, range trees, segment trees, and time forward processing. These in turn are subroutines for many external memory graph algorithms, such as expression tree evaluation, centroid decomposition, least common ancestor, minimum ....

[Article contains additional citation context not shown here]

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures, LNCS 955, pages 334--345, 1995. A complete version appears as BRICS technical report RS-96-28, University of Aarhus.

....can be performed in O(log B N B ) I Os, since only nodes on a single root to leaf path are visited by the (routing as well as the rebalancing) algorithm. Our technique for efficiently performing bulk operations on R trees is a variant of the general buffer treetechnique introduced by Arge [2, 3]. Here wemodify the general technique in a novel way, since a straightforward application of the technique would result in an R tree with an (impractically large) fan out of M=B [28] The main idea is the following: We attach buffers to all R tree nodes on every blog B M 4B cth level of the ....

....operations are performed by inserting them block by blockinto the buffer of the root. Buffer emptying is basically performed as discussed in the previous sections. The buffer emptying process for nodes on level blog B M 4B c have to be modified slightly and the interested reader is referred to [2] for details. As mentioned in the introduction, a buffering method similar to our approach has previously been presented byvan den Bercken, Seeger, and Widmayer [28] However, while our approach supports all kinds of bulk operations, the approach in [28] only supports bulk loading. To bulk load ....

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures, LNCS 955, pages 334--345, 1995. A complete version appears as BRICS technical report RS-96-28, Universityof Aarhus.

....compute (fA 1 ; Bg) and (fA 2 ; Bg) Queue fA 1 ; Bg with priority n( fA 1 ; Bg) and fA 2 ; Bg with priority n( fA 2 ; Bg) To bound the I O complexity of this procedure, observe that we queue a constant amount of information per edge in F. As there are O(N) edges in F , applying known results [11, 1] about time forward processing, we obtain the following lemma. Lemma 1. Given a set P of N points in R d , a fair split tree T of P with O(N) nodes and a separation constant s 0, a well separated realization of P can be computed in O(sort(N) I Os using O(N=B) blocks of external memory. ....

Lars Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proceedings of the Workshop on Algorithms and Data Structures, volume 955 of Lecture Notes in Computer Science, pages 334--345, 1995.

....results in this area have been obtained in [17, 32] Also, Kanellakis et al. 21] and Ramaswamy and Subramanian [26, 27] give efficient data structures for performing range searching in external memory. Very recently, a new data structure called buffer tree and its applications are given in [2, 3], and an external memory version of the directed topology tree ( 18] called topology B tree is given in [9] For excellent examples of experimental work in computational geometry, see Bentley [5, 6, 7, 8] As for experimental work on I O efficient computation, very recently Vengroff has built an ....

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures (to appear), 1995.

....becomes O(N logN ) Since for the applications we have in mind the number of spatiotemporal objects is large and cannot be kept in main memory, an external memory priority queue is needed. We propose using an implementation of an external memory priority queue that is based on the buffer tree [3]. The basic idea is to perform operations (insertions and deletions) off line in 17 such a way that the amortized complexity of each operation is O( 1 B logM B N B ) 4] As a result the running time of the algorithm in external memory becomes O( N B logM B N B ) I O s. Note that the ....

L. Arge. The Buffer Tree: A New Technique for Optimal I/O Algorithms. In Proc. Workshop on Algorithms and Data Structures, LCNS 955, pages 334-345, 1995. 29

....therefore follows that Step 2 of Algorithm 2 computes the desired separator. Now we show the I O complexity of Algorithm 2. The preorder numbering algorithm in [22] takes O(sort(N) I Os. All other vertex labels can be computed with O(sort(N) I Os using the time forward processing technique in [2, 1]. The lowest common ancestors for the vertices corresponding to each non tree edge can be computed with O(sort(N) I Os [2] because there are only O(N) edges. Copying the data from the vertices to the respective edges again takes a constant number of sorting and scanning passes over V 0 and E ....

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proceedings of the Workshop on Algorithms and Data Structures, volume 955 of Lecture Notes in Computer Science, pages 334--345, 1995.

....matching upper and lower I O bounds for several problems, and these bounds apply to the PDM model. The lower bound for sorting states that the worst case number of I O s required for sorting is Theta( N BD log M B N B ) 1 [3, 47] Several EM algorithms exist for sorting, including [2, 3, 4, 36, 37, 49, 50, 38]. Surprisingly, it turns out that performing a permutation requires Theta(minf N D ; N BD log M B N B g) I Os [3, 47] while the same can be performed in linear time in the RAM model. Similarly, the worst case number of I Os required to transpose a p Theta q matrix from row major ....

Arge, L. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures, LNCS 955 (1995), pp. 334--345. A complete version appears as BRICS technical report RS-96-28, University of Aarhus.

.... Gamma sort(E) Delta log log V B E Delta [25] MST O Gamma sort(E) Delta log V M Delta [12] O (sort(E) Delta log B scan(E) Delta log V ) 22] SSSP O Gamma V E B Delta log V B Delta [22] I O efficient graph algorithms have been considered by a number of authors [1, 2, 5, 6, 10, 12, 16, 19, 22, 24 26, 29]. Table 1 reviews the best known algorithms for basic graph theoretic problems on general undirected graphs. For directed graphs the best known algorithm for breadth first search (BFS) and depth first search (DFS) use O Gamma (V scan(E) Delta log V B sort(E) Delta I Os [10] Lower ....

....least one I O for each vertex which is included in the MST in order to read its adjacent vertices (traverse its adjacency lists) Thus processing all vertices and edges takes V E B I Os. It also performs O(E) insert s and extract min s on the priority queue. Using an external priority queue [5, 9] supporting these operations in O( 1 B log M=B N B ) I Os amortized we obtain: Lemma 1. The MST of an undirected weighted graph can be computed in O(V sort(E) I Os. 2.2 MST Vertex Reduction Algorithm Our MST vertex reduction algorithm is obtained using ideas from the connectedcomponent ....

[Article contains additional citation context not shown here]

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures, LNCS 955, pages 334--345, 1995. A complete version appears as BRICS technical report RS-96-28, University of Aarhus.

.... more complete view, we refer the interested reader to the excellent survey of Vitter [13] Among the general techniques proposed, we cite external sorting [1] matrix computations [14] external memory algorithms for computational geometry [6] for graph problems [3] and the buffer trees of Arge [2], a rather general technique which is able to trasform an internal memory tree data structure into an external memory structure. Another area in which the need for well engineered algorithms is crucial relates to numerical stable implementations. Indeed, developing robust and stable ....

L. Arge, "The buffer tree: a new technique for optimal I/O algorithms". Proc. 4th Workshop on Algorithms and Data Structures , Lecture Notes in Computer Science vol. 955, SpringerVerlag, 334--345, 1995. 4

....namely, that its resource consumption is within a constant factor of optimal. In Section 3, we present fundamental lower bounds for MA algorithms for sorting, standard) matrix multiplica1 tion, and related problems such as permuting, permutation networks, FFT computation, buffer tree operations [2], and LU factorization, with respect to our dynamic memory model. We obtain these lower bounds by suitably adapting the I O lower bounds [1, 7, 13] on conventional external memory algorithms to our dynamic memory setting. The lower bounds lead to natural definitions of resource consumption for all ....

....(3) The bounds in Theorem 1 lead naturally to the following problem specific (but algorithm independent)notion of the resource consumption by an MA algorithm: Definition 5 Consider the 2m I O operations of an MA algorithm A during any allocation phase of size m. 1 In the case of the buffer tree [2], N denotes the number of insert delete operations; further details are discussed in Appendix A. For sorting, FFT, and permutation networks, N is the number of input items. 2 Permuting refers to the special case of sorting in which the N keys are distinct integers in the range f1,2, Ng. 3 ....

[Article contains additional citation context not shown here]

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. Technical Report RS--94--16, BRICS, Univ. of Aarhus, Denmark, 1994.

.... External Memory (co authored with Vitter) OBDD] The I O Complexity of Ordered Binary Decision Diagram Manipulation [LowB] A General Lower Bound on the I O Complexity of Comparison based Algorithms (co authored with Knudsen and Larsen) Extended abstract versions of the papers have appeared in [11, 15, 16, 12, 13]. In the survey we refer to the papers as indicated above. The survey part of the thesis is divided into four chapters. In Chapter 2 we discuss basic paradigms for designing I O efficient algorithms and I O complexity of fundamental problems. This leads to a survey of external memory results in ....

....papers also deal with fundamental problems such as permutation, sorting and matrix transposition. The problem of implementing various classes of permutations has been addressed in [47, 48, 50] More recently researchers have moved on to more specialized problems in the computational geometry [11, 15, 34, 40, 67, 74, 79, 110, 121, 130, 137], graph [12, 40, 42, 97] and string areas [44, 56, 57] As already mentioned the number of I O operations needed to read the entire input is N=B and for convenience we call this quotient n. We use the term scanning to describe the fundamental primitive of reading (or writing) all elements in a ....

[Article contains additional citation context not shown here]

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures, LNCS 955, pages 334--345, 1995.

....to store segments in secondary structures such that queries can be answered efficiently. The starting idea behind our successful externalization of the structure, as compared with previous attempts [9, 35] is that the nodes in our structure have fan out p B instead of B, following ideas from [2, 3, 4]. The implications of this smaller fan out are explained later in this section. The external memory interval tree on a set of intervals I with endpoints in a fixed set E is defined in the following way (we assume without loss of generality that no two intervals in I have the same endpoint, and ....

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures, LNCS 955, pages 334--345, 1995. A complete version appears as BRICS technical report RS-96-28, University of Aarhus.

....are coming in from multiple processors. 1. 1 Summary of this Paper We present a new algorithm for the filter step called Scalable Sweeping Based Spatial Join (SSSJ) The algorithm uses several techniques for I O efficient computing recently proposed in computational geometry [APR 98, GTVV93, Arg95, AVV98, Arg97] plus the well known internal memory plane sweeping technique (see, e.g. PS85] It achieves theoretically optimal worst case bounds on both internal computation time and I O transfers, while also being efficient on the more wellbehaved data sets common in practice. We present ....

....will be used as a building block in the SSSJ algorithm described in the next section. We point out that this section is based on the results and theoretical framework developed in [APR 98] The algorithm uses the distribution sweeping technique developed in [GTVV93] and further developed in [Arg95, AVV98] Following Aggarwal and Vitter [AV88] we use the following I O model: We make the assumption that each access to disk transmits one disk block with units of data, and we count this as one I O operation. We denote the total amount of main memory by . We assume that we are given two sets ....

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures, LNCS 955, pages 334--345, 1995. A complete version appears as BRICS Technical Report RS-96-28, University of Aarhus.

....on performance for any of the experiments in this paper. The presentation of PQ given here assumes that the priorityqueue never grows larger than the amount of internal memory available. Note, however, that PQ can be modified to handle overflow gracefully by using an external priority queue [2, 9], and that it can also be combined with the partitioning step along one dimension that SSSJ performs in the case of an overflow of the interval data structure. We omit these details here since they are only needed for unusual worst case input distributions. One key feature of PQ is that it ....

L. Arge. The buffer tree: A new technique for optimal I/O-algorithms. In Proc. Workshop on Algorithms and Data Structures, LNCS 955, pages 334--345, 1995. A complete version appears as BRICS technical report RS-96-28, University of Aarhus.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC