J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. of IPPS Workshop on Input/Output in Parallel Computer Systems, 1993.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....of ways. The most important usage of this information, however, is to decide a parallel I O method based on access patterns (hints) provided by the application. By comparing the storage pattern and access pattern of a dataset, the MDMS can, for example, advise the HSS to perform collective I O [16] or prefetching [19] for this dataset. ffl It stores information about the pending access patterns. It utilizes this information in taking some global data movement decisions (e.g. file migration [38, 14] and prestaging [38, 14] possibly involving datasets from multiple applications. ffl It ....

....(Data set size is 2 MB) 4 procs 8 procs Original 2.27 1.34 Optimized 1.91 1.15 6 Related Work Numerous techniques for optimizing I O accesses have been proposed in literature. These techniques can be classified into three categories: the parallel file system and run time system optimizations [22, 8, 10, 19, 21, 16], compiler optimizations [4, 20, 17] and application analysis and optimization [20, 6, 29, 17, 7, 37] Brown et al. 5] proposed a meta data system on top of HPSS using DB2 DBMS. Our work, in contrast, focuses more on utilizing state of the art I O optimizations with minimal programming effort. ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel i/o via a two-phase run-time access strategy. In Proc. the 1993.

....individual requests of each process are noncontiguous. The merged request can therefore be serviced efficiently. Such optimization is broadly referred to as collective I O. Collective I O has been shown to be a very important optimization in parallel I O and can improve performance significantly [5, 14, 25, 30, 33]. Since none of the file systemson which ROMIO is implemented perform collective I O, ROMIO performs two phase collective I O on top of the file system. In the communication phase, interprocess communication is used to rearrange data into large chunks. In the I O phase, processes perform parallel ....

....actually writing data to the file, which is very expensive. 11. Leave Collective I O to the MPI IO Implementation. It is not entirely clear whether collective I O is better if performed in the file system or as a library above the file system. Both techniques have been proposed in the literature [5, 14, 25]. Our opinion is that, for implementing MPI IO s collectiveI O functionality, it is best if the file system focused on delivering the highest possible performance for independent (potentially noncontiguous) I O requests from individual processes (as mentioned in item 5 above) and the MPI IO ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved Parallel I/O via a Two-Phase Run-time Access Strategy. In Proceedings of the Workshop on I/O in Parallel Computer Systems at IPPS '93, pages 56--70, April 1993. Also published in Computer Architecture News, 21(5):31--38, December 1993.

....merging the requests of different processes and servicing the merged request. Such optimization is broadly referred to as collective I O. Collective I O can be performed at the disk level (diskdirected I O [5] at the server level (server directed I O [8] or at the client level (two phase I O [3]) Since ROMIO is a portable, user level library with no separate I O servers, it performs collective I O at the client level. For this purpose, it uses a generalized version of the extended twophase method described in [11] 4.1 Two Phase I O Two phase I O was first proposed in [3] in the ....

....I O [3] Since ROMIO is a portable, user level library with no separate I O servers, it performs collective I O at the client level. For this purpose, it uses a generalized version of the extended twophase method described in [11] 4. 1 Two Phase I O Two phase I O was first proposed in [3] in the context of accessing distributed arrays from files. Consider the example of reading a two dimensional array from a file into a (block,block) distribution in memory, as shown in Figure 2. Assume that the array is stored in the file in rowmajor order. As a result of the distribution in ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved Parallel I/O via a Two-Phase Run-time Access Strategy. In Proceedings of the Workshop on I/O in Parallel Computer Systems at IPPS '93, pages 56--70, April 1993. Also published in Computer Architecture News, 21(5):31--38, December 1993.

....and consequently, poor performance at the I O processors. Providing high level access pattern information to the file system through an application programming interface (API) allows disks to reorder requests, servicing them to maximize throughput. This motivates optimizations such as two phase [7] and disk directed I O [11] given global, high level knowledge that some data must be read or written before all the processors can proceed, the I O operations can be reordered to occur as an efficient collective. In recognition of its importance to high performance I O, an interface for ....

....participants to join the collective before initiating the transfer, or service each request immediately. There are several implementation alternatives for collective I O that can be categorized as user level or system level. User level libraries to support collective I O, such as two phase I O [7] have processors access files in long contiguous portions and permute the data among themselves to correspond to the processor data decomposition. Two major performance problems with this approach are that the user level library cannot exploit information about the physical disk layout, and the ....

[Article contains additional citation context not shown here]

DEL ROSARIO, J. M., BORDAWEKAR, R., AND CHOUDHARY, A. Improved Parallel I/O via a Two-Phase RunTime Access Strategy. In IPPS '93 Workshop on Input/Output in Parallel Computer Systems (1993), pp. 56--70. Also published in Computer Architecture News 21(5), December 1993, pages 31--38.

....several processors write to a common file in parallel. This would make better use of the available I O bandwidth and also eliminate the communication bottleneck. The I O access pattern of this application suggests that it would benefit from using collective I O, in the form of either two phase I O [10], disk directed I O [18] or serverdirected collective I O [31] In this application, arrays are distributed in a block fashion in two dimensions, resulting in each processor getting a sub block of the array. The sub block from each processor needs to written to the file containing the entire ....

....or serverdirected collective I O [31] In this application, arrays are distributed in a block fashion in two dimensions, resulting in each processor getting a sub block of the array. The sub block from each processor needs to written to the file containing the entire array. It has been shown in [10, 18, 31] that this can be done efficiently using collective I O. To study the benefits of using collective I O for this application, we have implemented the Chameleon I O routines PIWriteDistributedArray and PIReadDistributedArray using twophase I O. We are currently in the process of characterizing the ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved Parallel I/O via a Two-Phase Runtime Access Strategy. In Proceedings of the Workshop on I/O in Parallel Computer Systems at IPPS '93, pages 56--70, April 1993.

No context found.

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the 1993.

....Introduction Data intensive applications have presented challenging problems to computational scientists. A major problem is the I O performance when many datasets are stored and frequently accessed. Over the years, scientists have developed techniques for I O optimizations such as collective I O [14, 38, 23, 10], prefetching [15, 13] data sieving [38, 10] caching [8] and so on. But the performance problem still persists because data intensive applications are Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL 60208, choudhar ece.nwu.edu generating more data and ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel i/o via a two-phase run-time access strategy. In Proc. the 1993.

....during program execution. In this paper, we propose a technique called the Extended Two Phase Method, for accessing sections of out of core arrays efficiently. This is an extension and generalization of the Two Phase Method for reading in core arrays from files, which was previously proposed in [7, 3]. The Extended Two Phase Method uses collective I O in which all processors cooperate to perform I O in an efficient manner by combining several I O requests into fewer larger requests, eliminating multiple disk accesses for the same data and reducing contention for disks. We describe the ....

....the I O performance may be very low because of lower granularity of accesses and multiple accesses for the same data. Therefore, it is necessary to use a more efficient technique for doing I O in out of core computations. In the case of in core arrays, Bordawekar, del Rosario and Choudhary [7, 3] have proposed a Two Phase Method for reading an entire array from a file in an efficient manner, into a distributed array in main memory. This method is found to give consistently good performance for all data distributions. In this paper, we propose an Extended Two Phase Method for accessing ....

[Article contains additional citation context not shown here]

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved Parallel I/O via a Two-Phase Runtime Access Strategy. In Proceedings of the Workshop on I/0 in Parallel Computer Sys- tems at IPPS '93, April 1993.

....code reuse, and ease of programming. Notice also that al though the parallel I O libraries (e.g. 6] built on top of parallel file systems have the potential to provide both ease of use and high performance through the use of advanced I O optimization techniques such as col lective I O [14,20], and array chunking [26] their ex tensibility is severely limited by the design principles and the programming language used in their implemen tation [19] At the other end of the spectrum are database man agement systems (DBMS) They provide a layer on top of file systems, which is ....

....of ways. The most important usage of this informa tion, however, is to decide a parallel I 0 method based on access patterns (hints) provided by the application. By comparing the storage pattern and access pattern of a data set, the MDMS can, for example, advise the HSS to perform collective I O [14] or prefetching [17] for the data set in question. It stores information about the pending access pat terns. It utilizes this information in taking some global decisions, possibly involving data sets from multiple applications (e.g. staging a number of related files from tape sub system to ....

[Article contains additional citation context not shown here]

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the 1993.

....of ways. The most important usage of this information, however, is to decide a parallel I O method based on access patterns (hints) provided by the application. By comparing the storage pattern and access pattern of a dataset, the MDMS can, for example, advise the HSS to perform collective I O [17] or prefetching [20] for this dataset. ffl It stores information about the pending access patterns. It utilizes this information in taking some global data movement decisions (e.g. file migration [39, 13] and prestaging [39, 13] possibly involving datasets from multiple applications. ffl It ....

....prestaging, prefetching, and computation, thereby maximizing the I O performance. 7 Related Work Numerous techniques for optimizing I O accesses have been proposed in literature. These techniques can be classified into three categories: the parallel file system and run time system optimizations [24, 8, 10, 20, 22, 17], compiler optimizations [4, 21, 18] and application analysis and optimization [21, 7, 30, 18] Brown et al. 6] proposed a meta data system on top of HPSS using DB2 DBMS. Our work, in contrast, focuses more on utilizing state of the art I O optimizations with minimal programming effort. ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the 1993.

No context found.

Rosario, J., Bordawekar, R., Choudhary, A. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the 1993.

No context found.

Rosario, J., Bordawekar, R., Choudhary, A. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the 1993.

....3(a) where four processors are requesting four different sub columns of a region. The underlying cover contains 28 sub files. After bringing these sub files from tape to disk, we have a problem of reading the required sub portions (sub columns) for each processor. As stated by del Rosario et al. [20], collective I O is a technique in which processors perform I O on 64 000000000000000000 000000000000000000 000000000000000000 000000000000000000 000000000000000000 000000000000000000 000000000000000000 000000000000000000 000000000000000000 000000000000000000 000000000000000000 000000000000000000 ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the 1993.

....sub columns of a region. The underlying cover contains 28 sub files. After bringing these sub files from tape to disk, we have a problem of reading the required sub portions (sub columns) for each processor. We solve this problem by using multiple collective I O. As stated by del Rosario et al. [33], collective I O is a technique in which processors perform I O on behalf of each other in order to reduce the time spent in disk I O at the expense of some extra communication. Two phase I O is a specific implementation of collective I O, which uses the information available about the access and ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the

....1(c) where four processors are requesting four different sub columns of a region. The underlying cover contains 28 sub files. After bringing these sub files from tape to disk, we have a problem of reading the required sub portions (sub columns) for each processor. As stated by del Rosario et al. [9], collective I O is a technique in which processors perform I O 3 P P P P 0 1 2 3 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 (a) 0,0) 7,7) 1 1 requested 1 1 1 1 1 1 1 1 1 1 2 2 2 2 1 1 1 1 2 2 2 2 1 1 1 1 1 1 1 1 (d) e) 1 1 1 2 2 2 (b) c) cover portion ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the

....on the data transfer size. It is instructive to compare our results for Vesta with measurements done with other parallel le systems. The only other system that provides data decomposition like Vesta is the nCUBE system [12] Detailed measurements of various access patterns are presented in [13]. These measurements indicate that when an access includes transposition (e.g. access by rows to data that is stored by columns) the degradation in performance can be very large. Their proposed solution is to use two phase access. For example, a two phase read is implemented by rst accessing the ....

J. M. del Rosario, R. Bordawekar, and A. Choudhary, \Improved parallel I/O via a two-phase run-time access strategy". In Proc. IPPS '93 Workshop on I/O in Parallel Computer Systems, pp. 56-70, Apr 1993. (Reprinted in Comput. Arch. News 21(5), pp. 31-38, Dec 1993). 36

No context found.

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the 1993 IPPS Workshop on Input/Output in Parallel Computer Systems , April 1993.

....for improving I O performance of 27 I O intensive (not just out of core) parallel applications. Chameleon was the first runtime system which provided extensive support for parallel I O [GGL93] del Rosario et al. proposed a twophase access strategy for efficient access of distributed arrays [dRBC93, BdRC93] This strategy was later extended by Kotz to optimize disk accesses [Kot94] PASSION runtime system builds on [BdRC93] and provides runtime routines to access distributed multidimensional arrays in the Local as well as Global Placement Models [CBD 95] Similar runtime projects ....

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved Parallel I/O via a Two-Phase Runtime Access Strategy. In Proceedings of the Workshop on I/O in Parallel Computer Systems at IPPS '93, April 1993. 30

....could place a significant load on the running time of the computation. Even if the monitoring is going to be done off line, the volume of data required for playback of graphics can also affect running time adversely. Table 1 shows details of I O requirements for some Grand Challenge applications [dRBC93] In terms of the above discussion, temporary working storage generally comes from an out of core problem, archival and Electrical and Computer Engineering Department, Syracuse University, supported in part by NSF Young Investigator Award CCR 9357840, grants from Intel SSD and in part by the ....

....in the same distribution it is needed. For example, in an image processing application, images may arrive (even the network or from archives) in files. Each file will have to be redistributed into local files based on the out of core distribution. These routines use the two phase access strategy [dRBC93, BdRC93] which will read the data from the input data file using the most optimal access pattern (which depends on how the data is stored on disks) and redistribute the data over the processors using the high speed processor interconnection network. The processors will then store the data into ....

[Article contains additional citation context not shown here]

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase runtime access strategy. In Proceedings of the Workshop on I/O in Parallel Computer Systems at IPPS '93, April 1993.

....code reuse, and ease of programming. Notice also that although the parallel I O libraries (e.g. 6] built on top of parallel file systems have the potential to provide both ease of use and high performance through the use of advanced I O optimization techniques such as collective I O [14,20], and array chunking [26] their extensibility is severely limited by the design principles and the programming language used in their implementation [19] At the other end of the spectrum are database management systems (DBMS) They provide a layer on top of file systems, which is portable, ....

....parallel I O method based on access patterns (hints) provided by the application. By comparing the storage pattern and access pattern 4 A. Choudhary et al. Data Management for Large Scale Scientific Computations of a data set, the MDMS can, for example, advise the HSS to perform collective I O [14] or prefetching [17] for the data set in question. ffl It stores information about the pending access patterns. It utilizes this information in taking some global decisions, possibly involving data sets from multiple applications (e.g. staging a number of related files from tape sub system to ....

[Article contains additional citation context not shown here]

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. the 1993 IPPS Workshop on Input/Output in Parallel Computer Systems , April 1993.

No context found.

J. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proc. of IPPS Workshop on Input/Output in Parallel Computer Systems, 1993.

No context found.

J. M. Del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel i/o via two-phase run-time access strategy. ACM Computer Architecture News, 21(5):31-- 38, 1993.

No context found.

J. M. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proceedings of the IPPS '93 Workshop on Input/Output in Parallel Computer Systems, pages 56--70, Newport Beach, CA, 1993.

No context found.

J. M. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proceedings of the IPPS '93 Workshop on Input/Output in Parallel Computer Systems, pages 56--70, Newport Beach, CA, 1993.

No context found.

J. M. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel I/O via a two-phase run-time access strategy. In Proceedings of the IPPS '93 Workshop on Input/Output in Parallel Computer Systems, pages 56--70, Newport Beach, CA, 1993.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC