D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proc. IEEE Symp. on Parallel and Distributed Computing, 1995.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....external memory [CGG 95, Arg96b, UY91] Various theoretical results have been obtained in the last years starting with classical sorting and searching problems [AV88] Very recently a few researchers also considered practical implementations. One of the first external memory libraries was TPIE [VV95] TPIE consists of several so called external programming paradigms like scanning, sorting and merging. The main drawback of TPIE is that there exists no support to internal memory libraries [HMSV97] Whenever internal data structures or algorithms are needed they must be explicitly implemented. ....

D.E. Vengroff and J.S. Vitter. I/O-efficient scientific computation using tpie. In IEEE Symposium on Parallel and Distributed

....all implemented in TPIE. This part of TPIE also contains fundamental data structures such as queues and stacks, algorithms for sorting and matrix operations, as well as a few more specialized geometric algorithms. It has been used in I O efficient implementations of several scientific computation [150], spatial join [24, 25, 23] and terrain flow [27, 19] algorithms. Since most external data structures cannot efficiently be implemented in a stream based framework, the second part of the TPIE project adds kernel support for a block oriented programming style. Like in LEDA SM, the external ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, 1996.

....area of relational databases, but mainly refers to spatial and temporal data. For spatial databases, for example, we provide several implementations of spatial join algorithms [3] The cursor based processing is however the major advantage of XXL in contrast to approaches like LEDA [6] and TPIE [7]. For more information on XXL see http: www.mathematik.uni marburg.de DBS xxl. We will demonstrate the latest version of XXL using examples to show its core functionality. We will concentrate on three key aspects of XXL. Usage: We show how easily state of the art spatial join algorithms can ....

D. E. Vengroff, J. S. Vitter. I/O-Efficient Scientific Computation using TPIE. Proc. Goddard Conference on Mass Storage Systems and Technologies, 1996, in NASA Conference Publication

.... be found in database systems [79, 110] spatial databases and geographic information systems (GIS) 52, 70, 84, 114, 124] VLSI verification [17] constraint logic programming [78, 79] computer graphics and virtual reality systems [60, 114] computational biology [135] physics and geophysics [47, 129] and in meteorology [47] The amount of data manipulated in such applications is too large to fit in main memory and must reside on disk, hence the I O communication can become a very severe bottleneck. A good example is NASA s EOS project GIS system [52] which is expected to manipulate petabytes ....

....the most essential parameters of the I O systems in use today, and theoretical results in the model can help to gain valuable insight. This is supported by experimental results which show that implementing algorithms designed for the model can lead to significant runtime improvements in practice [40, 41, 126, 129]. Finally, it should be mentioned that several authors have considered extended theoretical models that try to model the hierarchical nature of the memory of real machines [1, 2, 3, 4, 7, 77, 116, 131, 132, 134] but such models quickly become theoretically very complicated due to the large ....

[Article contains additional citation context not shown here]

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proc. IEEE Symp. on Parallel and Distributed Computing, 1995. Appears also as Duke University Dept. of Computer Science technical report CS-1995-18.

....Vitter and Shriver did not design their out of core algorithm for general permuting to be performed in place. Furthermore, the constants before the upper bound for this algorithm are rather high compared to the constants in the worst cases for the structured permutation algorithms mentioned above [VV95] The next two sections present out of core in place algorithms to perform several commonly used structured permutations. We supply exact constants for the I O complexity of both the previous best algorithms and the in place algorithms. With exact constants, we can examine the tradeoffs between ....

Darren Erik Vengroff and Jeffrey Scott Vitter. I/O-efficient scientific computation using TPIE. In Seventh IEEE Symposium on Parallel and Distributed Processing, October 1995. To appear. 21

....memory and slower external memory (disk) becomes a major bottleneck. Algorithms specifically designed to reduce the I O bottleneck are called external memory algorithms. Although most of the results in this area of research are theoretical, the experiments of Chiang [3] and of Vengroff and Vitter [19] on some of these techniques show that they result in significant improvements over traditional algorithms in practice. Also, Teller et al. 18] describe a system to compute radiosity solutions for polygonal environments larger than main memory, and Funkhouser et al. 7] present prefetching ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proc. IEEE Symp. on Parallel and Distributed Computing, 1995.

....it uses (in units of disk blocks) and the internal memory computation time. 1 More sophisticated measures of disk performance involve analysis of seek and rotational latencies and caching issues [25]# however, the simpler standard model has proven quite useful in identifying first order effects [31]. I O efficiency has always been a key issue in database design, but has only recently become a central area of investigation in the algorithms community.Aggarwal and Vitter [1] developed matching upper and lower I O bounds for a variety of fundamental problems such as sorting and permuting. For ....

....Subsequently, I O efficient algorithms have been developed for several problem domains, including computational geometry, graph theory, and string processing. Refer to recent surveys for references [3, 4, 32] The practical merits of the developed algorithms have been explored byanumber of authors [11, 31, 6, 5]. 1 For simplicitywe concentrate on the two first measures in this paper. It can be shown that the asymptotic internal memory computation time of our new R tree algorithms is the same as for the traditional algorithms. Much of this work uses the Transparent Parallel I O programming Environment ....

[Article contains additional citation context not shown here]

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, 1996.

.... of experimental work in computational geometry, see Bentley [5, 6, 7, 8] As for experimental work on I O efficient computation, very recently Vengroff has built an environment called TPIE for programming external memory algorithms as he proposed earlier in [29] and also Vengroff and Vitter [30] have reported some benchmarks of TPIE on sorting and matrix multiplication. This work, however, is mainly on providing a programming environment and not on performance comparisons between external memory algorithms and conventional algorithms. Other than this, we do not know of any previous ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. Manuscript, 1995.

.... 28, 44] recently various researchers have been investigating external memory algorithms for graphs [1, 12] and for computational geometry [1, 3, 5, 6, 7, 10, 18, 21, 29, 38, 43] As mentioned before, most of the results are theoretical, and yet the experiments of Chiang [11] Vengroff and Vitter [42], and Arge et al. 5] on some of these techniques show that they result in significant improvements over traditional algorithms in practice. As for isosurface extraction, there is a very rich literature. Here we only briefly review the results that focus on speeding up the search phase. We refer ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proc. IEEE Symp. on Parallel and Distributed Computing, 1995.

.... external memory algorithms for graphs [10] and for computational geometry [2, 4, 5, 8, 13, 16, 23, 31, 34] in addition to early work on sorting and scientific computing [1, 21, 35] Although most of the results are theoretical, the experiments of Chiang [9] and of Vengroff and Vitter [33] on some of these techniques show that they result in significant improvements over traditional algorithms in practice. Also, Teller et al. 32] describe a system to compute radiosity solutions for polygonal environments larger than main memory, and Funkhouser et al. 12] present prefetching ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proc. IEEE Symp. on Parallel and Distributed Computing, 1995.

.... experimental work in computational geometry, see Bentley [7, 8, 9, 10] As for experimental work on I O efficient computation, concurrent to our work Vengroff builds an environment called TPIE for programming external memory algorithms as he proposed earlier in [36] and also Vengroff and Vitter [37] report some benchmarks of TPIE on sorting and matrix multiplication. This work, however, is mainly on providing a programming environment and not on performance comparisons between external memory algorithms and conventional algorithms. Also worth noting is the work by Ramaswamy and Kanellakis ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proc. IEEE Symp. on Parallel and Distributed Computing, 1995.

....array, the partial sums along each row can be computed efficiently, but accessing the array column wise will cause almost one page fault per element accessed. To achieve optimal I O efficiency even when virtual memory is used, we can chunk the array, as suggested by [20, 19] and implemented in [23, 26] for other applications. Chunking is a way to divide a d dimensional array into d dimensional 35 D1 D3 1 2 3 4 5 9 10 11 12 26 25 55 56 D2 7 21 49 53 39 37 33 19 20 17 18 23 24 31 32 59 60 64 63 51 52 27 28 Figure 1: A chunked three dimensional array with each chunk consisting of 2 Theta 2 ....

....disk. Figure 1 shows a chunked three dimensional array. The size of a d dimensional chunk is c1 Theta Delta Delta Delta Thetac d = B where c i is the chunk size along dimension D i and B is the size of a disk block. To implement the chunked array, we could use an approach similar to those of [23, 26], in which the disk layout is explicitly managed. For example, the high level yet efficient TPIE system is used in [23] to do the chunking for matrix multiplication. In this paper, for simplicity, we avoid the need for a separate disk management system like TPIE and use the virtual memory system ....

[Article contains additional citation context not shown here]

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, 553--570, College Park, MD, September 1996.

.... above any native file system to provide a common application interface [Huber et al. 1995; Moyer and Sunderam 1994] Also, there have been several projects to develop libraries and servers specifically targeted to meet the needs of scientific applications [Seamons et al. 1995; Thakur et al. 1994; Vengroff and Vitter 1995]. The flexibility of HFS allows it to be easily extended to support new interface standards as they are 30 Delta O.Krieger and M.Stumm developed. Moreover, because HFS supports common I O interfaces, we expect it to be a simple target for libraries and servers that use native file systems for ....

Vengroff, D. E. and Vitter, J. S. 1995. I/O-efficient scientific computation using TPIE. In Proc. of the 1995 IEEE Symposium on Parallel and Distributed Processing (Oct. 1995), pp. 74--77.

.... EM algorithms, including those based upon the data stream model of computation [11] An even simpler approach with no duplication of blocks is possible for the important subclass of the class of multipass algorithms that are based upon the stream, distribution and distribution sweeping paradigms [16, 17]. For these algorithms, the RCD method works almost exactly as described for distribution sort, and the same analysis applies. No duplicate copies of blocks are needed. Relevant algorithms include orthogonal segment line segment intersection, all nearest neighbors of a point set and a variety of ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the 1995.

....synthetic datasets are generated using our own data generation model described in the Appendix. 7.2. Efficiency of the Compact Data Cube Construction Algorithm We implemented our compact data cube construction algorithms using the Transparent Parallel I O Programming Environment (TPIE) system [VV96, Ven97, Ven94] TPIE is a collection of templated functions and classes to support high level and efficient implementations of external memory algorithms. The basic data structure in TPIE is a stream, representing a list of objects of an arbitrary type. The system includes I O efficient ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, College Park, MD, September 1996.

....Partitioning with partitions and tiles using the round robin scheme. The rectangle drawn with a solid line will appear in all three partitions. 7. 2 Implementation Details We implemented the three algorithms using the Transparent Parallel I O Programming Environment (TPIE) system [Ven94, Ven95, VV96] see also http: www.cs.duke.edu TPIE ) TPIE is a collection of templated functions and classes to support high level yet efficient implementations of external memory algorithms. The basic data structure in TPIE is a stream, representing a list of objects of an arbitrary type. The system ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, 1996.

....with 3 partitions and 12 tiles using the round robin scheme. The rectangle drawn with a solid line will appear in all three partitions. 7. 2 Implementation Details We implemented the three algorithms using the Transparent Parallel I O Programming Environment (TPIE) system [Ven94, Ven95, VV96] see also http: www.cs.duke.edu TPIE ) TPIE is a collection of templated functions and classes to support high level yet efficient implementations of external memory algorithms. The basic data structure in TPIE is a stream, representing a list of objects of an arbitrary type. The system ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, 1996.

....the same experimental platform on all machines we decided to assign two (consecutive) pages to each node on Machine 1, that is, to request 8 KB per I O operation. 5. 2 Software Environment We implemented the algorithms in C using the Transparent Parallel I O Programming Environment (TPIE) [41, 40, 42]; see also http: www.cs.duke.edu TPIE . TPIE is a C templated library that supports high level, yet efficient implementations of external memory algorithms. The basic data structure in TPIE is a stream, representing a list of objects of an arbitrary type. TPIE includes implementations of ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, 1996. 20

....partition is loaded into memory and intersections between rectangles in the partition reported. This step was described in detail in Section 4.4. 5. 2 Implementation details We implemented the three algorithms using the Transparent Parallel I O Programming Environment (TPIE) system [Ven94, Venns, VV96] TPIE is a collection of templated functions and classes to support highlevel yet efficient implementations of external memory algorithms. The basic data structure in TPIE is a stream, representing a list of objects of an arbitrary type. The system contains I O efficient implementations of ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, College Park, MD, September 1996.

....data be discretized so that each attribute has a reasonable number of distinct values. If so, operations like histogram formation, which have a significant impact on performance, can be done in a linear number of I Os, usually requiring one, but never more than two passes over the DETAIL table [29]. Without the discretization, the I O performance bound has an extra factor that is logarithmic but fortunately with a very large base M=B, which is the number of disk blocks that can fit in the internal memory. One advantage of our approach is that its implementation is easy. We have implemented ....

....performance. If the dimension tables cannot fit in memory, they can be formed by sorting in linear time, if we make the weak assumption that (M=B) c D=B for some small positive constant c, where D, M , and B are respectively the dimension table size, the internal memory size, and the block size [4, 29]. This optimization can be obtained automatically if SQL has the multiple M size of internal memory B size of disk block N # of rows in DETAIL n # of attributes in DETAIL (not including class label) C # of distinct class labels L depth of the final classifier D k total size of all ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, College Park, MD, September 1996.

....synthetic datasets are generated using our own data generation model described in the Appendix. 7.2. Efficiency of the Compact Data Cube Construction Algorithm We implemented our compact data cube construction algorithms using the Transparent Parallel I O Programming Environment (TPIE) system [VV96, Ven97, Ven94] TPIE is a collection of templated functions and classes to support high level and efficient implementations of external memory algorithms. The basic data structure in TPIE is a stream, representing a list of objects of an arbitrary type. The system includes I O efficient ....

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proceedings of the Goddard Conference on Mass Storage Systems and Technologies, NASA Conference Publication 3340, Volume II, pages 553--570, College Park, MD, September 1996.

No context found.

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proc. IEEE Symp. on Parallel and Distributed Computing, 1995.

No context found.

D. E. Vengroff and J. S. Vitter. I/O-efficient scientific computation using TPIE. In Proc. IEEE Symp. on Parallel and Distributed Computing, 1995.

No context found.

D.E. Vengroff and J.S. Vitter. I/o-efficientscientific computation using tpie. In Proc. GoddardConference on Mass Storage Systems and Technologies, pages 553-- 570, 1996.

No context found.

D.E. Vengroff and J.S. Vitter. I/o-efficient scientific computation using tpie. In Proc. Goddard Conference on Mass Storage Systems and Technologies, pages 553-- 570, 1996.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC