Rajeev Thakur, William Gropp, and Ewing Lusk. On Implementing MPI-IO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....to the performance problems of her applications; second, making decisions among various optimization candidates requires expert knowledge, and therefore, it is not a trivial task. Unfortunately, the existing I O systems do require users with such kind of expertise. For instance, the MPI IO [38, 37] provides more than 30 I O functions, how to choose a suitable function among them is not an easy task. Therefore, the decision of I O optimizations should be left to the I O system itself. On the other hand, the user has the best knowledge of how her dataset will be stored and accessed, so user s ....

R. Thakur, W. Gropp, and E. Lusk. On implementing MPI-IO portably and with high performance. Preprint ANL/MCS-P7321098, Mathematics and Computer Science Division, Argonne National Laboratory, 1998.

....high performance parallel I O. Details about these features can be found in [9] Network Remote Site ADIO ADIO MPI IO File System specific Implementations Portable Implementations Intel IBM SGI HP PFS PIOFS XFS HFS Unix NFS NEC SFS Figure 3. ROMIO architecture (from [15]) and our approach Multiple implementations of MPI IO, both portable and machine specific, are available. ROMIO [16] is a wellknown implementation of MPI IO with high performance and portability on different platforms [16, 15] ROMIO has a key internal layer called ADIO: an abstract device ....

....PFS PIOFS XFS HFS Unix NFS NEC SFS Figure 3. ROMIO architecture (from [15] and our approach Multiple implementations of MPI IO, both portable and machine specific, are available. ROMIO [16] is a wellknown implementation of MPI IO with high performance and portability on different platforms [16, 15]. ROMIO has a key internal layer called ADIO: an abstract device interface for parallel I O, which consists of a small set of basic functions for parallel I O. ROMIO achieves portability and high performance for MPI IO by implementing MPI IO on top of ADIO, and implementing ADIO separately on each ....

[Article contains additional citation context not shown here]

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPIIO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

....of porting a program from one machine to another a very difficult task. Third, the file system policies and optimization parameters are in general hard coded within the file system and, consequently, work for only a small set of access patterns. While runtime systems and libraries like MPI IO [10, 34] and others [36, 3, 8] present users with higher level, more structured interfaces, the excessive number of calls to select from, each with several parameters, make the user s job very difficult. Also, the usability of these libraries depends largely on how well user s access patterns and library ....

R. Thakur, W. Gropp, and E. Lusk. On implementing MPI-IO portably and with high performance. Preprint ANL/MCS-P732-1098, Mathematics and Computer Science Division, Argonne National Laboratory, 1998.

....have been successful in improving performance. Many parallel applications require access to small, potentially non contiguous, data chunks. Collective I O merges multiple I O accesses (both contiguous and noncontiguous) generated by multiple processes, into a single I O function call. MPI I O [26, 28], which is included as part of the MPI 2 standard release, provides for collective communication so that multiple MPI processes can access a single MPI I O file. We utilize MPI collective I O (we will refer to collective I O as MPI I O in this paper) in the work presented here. But when multiple ....

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPI-IO Portably and with High Performance. In Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pages 23--32, 1999.

....The user can use common file system commands such as cp, mkdir, ls etc to operate DPFS files and directories. In the future, we will use DPFS for some real world applications such as astrophysics application and use DPFS as a low level system to service a high level interface such as MPI I O [24] and MDMS [18] Acknowledgments This research was in part supported by Department of Energy under the Accelerated Strategic Computing Initiative (ASCI) Academic Strategic Alliance Program (ASAP) Level 2, under subcontract No W 7405 ENG 48 from Lawrence Livermore National Laboratories. We would ....

R. Thakur, W. Gropp, and E. Lusk. On implementing MPI-IO portably and with high performance. Preprint ANL/MCS-P7321098, Mathematics and Computer Science Division, Argonne National Laboratory, 1998.

....background threads, our proposed approach makes no assumptions regarding buffer space availability, with a buffering scheme automatically adjusting to available memory space and applications output patterns. 3. We implemented ABT in a widely used and supported parallel MPI IO library, ROMIO [14], in such a way that the buffering and background I O are transparent to the users and the collective output interfaces remain unchanged. Further, ABT is implemented in the file system independent layer of ROMIO, allowing it to be easily ported and to provide extra performance gain on top of ....

....to be within 70 of the local memory copy bandwidth when sufficient space is available on the compute processors to buffer all the output, and within 90 of the MPI bandwidth for any portion of the output that overflows client buffers and fits into server buffers [5] 2. 2 The ROMIO library ROMIO [14], developed at the Argonne National Laboratory, is a well known implementation of the MPIIO specification in the MPI 2 standard [6] As part of the popular MPI implementation MPICH, it supports a wide range of file systems, and has an intermediate interface called the ADIO (Abstract Device ....

R. Thakur, W. Gropp, and E. Lusk. On implementing MPI-IO portably and with high performance. In Proceedings of the Sixth Workshop on I/O in Parallel and Distributed Systems, May 1999. 10

No context found.

R. Thakur, W. Gropp, and E. Lusk. On implementing MPIIO portably and with high performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

....MPI IO and HDF, are most closely related to our research. MPI IO is a parallel I O interface specified in the MPI 2 standard. It is implemented and used on a wide range of platforms. The most popular implementation, ROMIO [17] is implemented portably on top of an abstract I O device layer [14, 16] that enables portability to new underlying I O systems. One of the most important features in ROMIO is collective I O operations, which adopt a two phase I O strategy [11, 12, 13, 15] and improve the parallel I O performance by significantly reducing the number of I O requests that would ....

R. Thakur, W. Gropp, and E. Lusk. "On implementing MPIIO portably and with high performance," in Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, May 1999, pp. 23-32.

....the readv and writev calls to avoid the need for data copy. In these cases we instead convert the type to a list of (offset, length) pairs to be passed to these calls. In this case too we must partial process, as these calls will only accept a limited number of these pairs. Finally the ROMIO [10] MPI IO component of MPICH2 requires similar (offset, length) pairs for use with noncontiguous file views. The sizes of these types do not match the sizes of the types for readv and writev calls on all platforms, so separate routines are required. From this it is obvious that just in the context ....

R. Thakur, W. Gropp, and E. Lusk. On implementing MPI-IO portably and with high performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

No context found.

Rajeev Thakur, William Gropp, and Ewing Lusk. On Implementing MPI-IO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

No context found.

R. Thakur, W. Gropp, and E. Lusk. On implementing MPIIO portably and with high performance. In Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pages 23--32, May 1999.

No context found.

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPIIO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

No context found.

Rajeev Thakur, William Gropp, and Ewing Lusk. On Implementing MPI-IO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23-- 32. ACM Press, May 1999.

No context found.

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPI-IO Portably and with High Performance. In the 6th Workshop on I/O in Parallel and Distributed Systems, 1999.

No context found.

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPIIO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

No context found.

Rajeev Thakur, William Gropp, and Ewing Lusk, On Implementing MPI-IO Portably and with High Performance, in Proceedings of the Sixth Workshop on I/O in Parallel and Distributed Systems, May 1999, pages 23--32.

No context found.

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPIIO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

No context found.

Rajeev Thakur, William Gropp, and Ewing Lusk. On Implementing MPI-IO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23-- 32. ACM Press, May 1999.

No context found.

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPI-IO Portably and with High Performance. In Proceedings of the Sixth Workshop on Input/Output in Parallel a nd Distributed Systems, pages 23--32, 1999.

No context found.

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPIIO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999. 12

No context found.

R. Thakur, W. Gropp, and E. Lusk. On Implementing MPIIO Portably and with High Performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

No context found.

Rajeev Thakur, William Gropp, and Ewing Lusk. On implementing MPI-IO portably and with high performance. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pages 23--32. ACM Press, May 1999.

No context found.

Rajeev Thakur, William Gropp, and Ewing Lusk. On implementing MPI-IO portably and with high performance. In Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pages 23--32, May 1999.

No context found.

R. Thakur, W. Gropp, and E. Lusk, "On Implementing MPI-IO Portably and with High Performance", in Proc. Of the Sixth Workshop on I/O in Parallel and Distributed Systems, May 1999, pages 23-32.

No context found.

R. Thakur, W. Gropp and E. Lusk, On implementing MPI-IO portably and with high performance, Preprint ANL/MCS-P732-1098, Argonne National Laboratory, Mathematics and Computer Science Division (1998).

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC