A.S. Grimshaw, J. Prem, High performance parallel file objects, in: Proceedings of the Sixth Annual Distributedmemory Computer Conference, 1991, pp. 720--723.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... encourages the development of application specific libraries that provide the interface and features that benefit applications [15] For example, tailoring the prefetching and caching policies to match the application s access patterns can reduce latency and avoid unnecessary data requests [16 18], and matching data distribution policies to the application s access patterns can optimize parallel access to distributed disks [19,20] 1.2. Remote processing of application code A typical data intensive grid application consists of a set of client processes and a set of remote data servers. ....

A.S. Grimshaw, J. Prem, High performance parallel file objects, in: Proceedings of the Sixth Annual Distributedmemory Computer Conference, 1991, pp. 720--723.

....memory (DSM) system, consistency protocols may need adjustment (since they are normally designed for virtual memory access patterns) Indeed, many operating systems for distributed memory machines do not support DSM, and thus could not easily support memory mapped files. Grimshaw, Loyot, and Prem [13] outline an extensible object oriented interface based on a simple low level, Unix like file system interface. The objectoriented front end encapsulates access methods, caching, prefetching, and file layout in applicationspecific ways. They focus on providing the mechanism without specifying ....

Andrew S. Grimshaw and Jeff Prem. High performance parallel file objects. In Sixth Annual Distributed-Memory Computer Conference, pages 720--723, 1991.

....memory (DSM) system, consistency protocols may need adjustment (since they are normally designed for virtual memory access patterns) Indeed, many operating systems for distributed memory machines do not support DSM, and thus could not easily support memorymapped files. Grimshaw, Loyot, and Prem [GP91, GL91] outline an extensible object oriented interface based on a simple low level, Unix like file system interface. The object oriented front end encapsulates access methods, caching, prefetching, and file layout in application specific ways. They focus on providing the mechanism without ....

Andrew S. Grimshaw and Jeff Prem. High performance parallel file objects. In Sixth Annual Distributed-Memory Computer Conference, pages 720--723, 1991.

....allow the programmer to request noncontiguous data in a single request, there is no support for collective I O. CM Fortran for the CM 5 does provide a collective I O interface, which leads to high performance through cooperation among the compiler, runtime, operating system, and hardware. ELFS [GP91] provides an object oriented interface that encourages operations on large objects, and could lead to support for collective I O. Finally, there are several interfaces for collective matrix I O [GGL93, BdC93, BBS 94] For example, to 1 This scenario arises in many situations. The file may ....

Andrew S. Grimshaw and Jeff Prem. High performance parallel file objects. In Sixth Annual Distributed-Memory Computer Conference, pages 720--723, 1991.

....Size Broadcast Access Pattern 8 IOPs 4 IOPs 16 IOPs 32 IOPs Figure 8: Increase in throughput for read requests using the strided interface. Note the different scales on the y axis. underlying (disk, record) pair, and also allows an application to provide its own mapping function. The ELFS system [GP91] and the Hurricane File System [Kri94] provide object oriented interfaces. These interfaces allow library designers to implement complex functionality (e.g. transparent replication of data, applicationspecific caching algorithms) in their files, but to hide that complexity from end users. The ....

Andrew S. Grimshaw and Jeff Prem. High performance parallel file objects. In Sixth Annual Distributed-Memory Computer Conference, pages 720--723, 1991.

....without physically changing the file s organization. Previous Work One early implementation is the Intel Concurrent File System [4] Crockett [1] outlines a multiprocessor file system design. The most exciting recent work is the new nCUBE file system [2] and the ELFS object oriented interface [3]. ....

A. S. Grimshaw and J. Prem. High performance parallel file objects. In Proceedings of the Sixth Annual Distributed-Memory Computer Conference, pages 720--723, 1991.

....file declustering, caching, and prefetching. The performance of Intel s CFS when reading or writing a two dimensional matrix, for example, depends heavily on the layout of the matrix across disks and across memories of the multiprocessor, and also on the order of requests [dBC93, BCR92, Nit92, GP91, GL91] del Rosario et al. dBC93] find that the nCUBE exhibits similar inefficiencies: when reading columns from a two dimensional matrix stored in row major order, read times increase by factors of 30 50. One solution is to transfer data from disk into memory and then permute it within memory ....

....to the disks by reducing concurrency [Nit92] Each of these examples highlights the need for programs to organize their I O carefully. To do so, we must be able to discover and control the I O system configuration. Grimshaw et al. make many of the same arguments for their ELFS file system [GP91, GL91] ELFS is an extensible file system, building object oriented, operation specific classes on top of a simple set of file access primitives. ELFS leaves decisions about declustering, caching, and prefetching to the higher level functions, which have a broader understanding of the operation. ....

Andrew S. Grimshaw and Jeff Prem. High performance parallel file objects. In Sixth Annual Distributed-Memory Computer Conference, pages 720--723, 1991.

....look good on paper will not work in practice. It appears that MPI IO could also feasibly be implemented on top of a nested batched interface. 8.2. 5 ELFS The ELFS system, from the University of Virginia, is an object oriented file system that has tight ties to the Mentat programming language [GP91, GL91] Files in ELFS are instances of object classes, which provide a high level interface to an abstract data structure and encapsulate the access patterns and the actual structure of the file. Each object has a separate thread of control, allowing them to perform asynchronous data transfers ....

Andrew S. Grimshaw and Jeff Prem. High performance parallel file objects. In Sixth Annual Distributed-Memory Computer Conference, pages 720--723, 1991.

....file declustering, caching, and prefetching. The performance of Intel s CFS when reading or writing a two dimensional matrix, for example, depends heavily on the layout of the matrix across disks and across memories of the multiprocessor, and also on the order of requests [dBC93, BCR93, Nit92, GP91, GL91] del Rosario et al. dBC93] find that the nCUBE exhibits similar inefficiencies: when reading columns from a two dimensional matrix stored in row major order, read times increase by factors of 30 50. One solution is to transfer data from disk into memory and then permute it within memory ....

....in terms of the number and order of disk accesses [Kot94] Each of these examples highlights the need for programs to organize their I O carefully. To do so, we must have file system primitives to discover and control the I O system configuration. The ELFS file system is based on this principle [GP91, GL91] ELFS is an extensible file system, building object oriented, application specific classes on top of a simple set of file access primitives. ELFS leaves decisions about declustering, caching, and prefetching to the higher level functions, which have a broader understanding of the ....

Andrew S. Grimshaw and Jeff Prem. High performance parallel file objects. In Sixth Annual Distributed-Memory Computer Conference, pages 720--723, 1991.

....variable consistency files can be used to avoid the consistency semantics of NFS files [9] 2.2.1 Parallel File Objects pfo s Parallel file objects attack both the bandwidth and latency problems on parallel and distributed systems. A detailed description, including performance, can be found in [6]. In addition to the unix file operators, pfo s provide operators that allow the user to: 2. The distinction between independent and contained objects is not unusual, and is driven by efficiency considerations. 6 1) specify the structure of the file, e.g. 1D array, 2D array, 3D array. This has ....

A. S. Grimshaw, and J. Prem, `High Performance Parallel File Objects,' To appear in 6th Distributed Memory Computing Conference, Portland, OR., April 1991.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC