Peter Brezany, Michael Gernt, Piyush Mehotra, and Hans Zima. Concurrent file operations in a High Performance FORTRAN. In Proceedings of Supercomputing '92, pages 230--237, 1992. 99  Home/Search   Document Details and Download   Summary   Related Articles   Check

....for multifiles , which are too closely tied to the number of physical processors being employed [3, 5] Virtual file pointers also maximize asynchrony, which is very important for efficiency on MIMD hardware. Systems that allow i o operations on parallel aggregates, but with a single file pointer [1, 10], force over synchronization of many applications. This work is being supported by National Science Foundation grants ASC 9203971 and CDA 9421997. 1 Since we are simply allowing the C VPs to execute the standard Unix i o functions, the function interface is nearly identical to that used by ....

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent file operations in a high performance FORTRAN. Technical Report 92--46, Institute for Computer Applications in Science and Engineering, NASA Langley Research Center, 1992.

....parallel computers have become very popular in the scientific community. One of the main reasons for their success, is the availability of efficient and simple parallel languages. MIMD languages like Fortran D [Fox90, Fox91, FHK 90, Tse93] Vienna Fortran [CMZ92, BGMZ92a, ZBC 92, BCZ92, BGMZ92b] and Fortran90D HPF [Zek92c, Zek92b, For93] which use data decompositions to achieve concurrency. Data decomposition often requires the data to be accessed (read written) from the disks in a distributed manner. In recent past, various attempts were made to develop languages that would support the ....

....the array, A, above could be written into a file, f, by using the following statement CWRITE(f, PROCESSORS= P(2,2) DIST= BLOCK,BLOCK) TO P ) A CREAD is used in the similar fashion. Other parallel I O statements include COPEN, CCLOSE, CSKIP etc.For further detail, the reader is referred to [BGMZ92b] 5.1.2 High Performance Fortran High Performance Fortran or HPF was designed to provide language support for a variety of machines including SIMD, MIMD, and vector machines. The HPF is syntactically similar to Fortran D [FHK 90, Fox90, Fox91] however, HPF incorporates many more features ....

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent File Operations in a high Performance Fortran. In Supercomputing 92, pages 230--238, November 1992.

....Languages In recent years, parallel computers have become very popular in the scientific community. One of the main reasons for their success, is the availability of efficient and simple parallel languages. MIMD languages like Fortran D [Fox90, Fox91, FHK 90, Tse93] Vienna Fortran [CMZ92, BGMZ92a, ZBC 92, BCZ92, BGMZ92b] and Fortran90D HPF [Zek92c, Zek92b, For93] which use data decompositions to achieve concurrency. Data decomposition often requires the data to be accessed (read written) from the disks in a distributed manner. In recent past, various attempts were made to develop ....

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent File Operations in a high Performance Fortran. Technical Report ICASE Report 92-46, ICASE, NASA, Hampton VA 23681, September 1992.

....that meet the needs of the applications we are supporting. Current research efforts in compiler technology for massively parallel machines are addressing the issue of distributing large multidimensional arrays across multiple processors. In High Performance Fortran and related languages [HPF93, Fox90, Hiranandani92, Brezany92], the user can provide hints to the compiler, with the ALIGN and DISTRIBUTE commands, that are used to create a data layout and problem decomposition across the processors that is appropriate for the computation. Except for the hints to the compiler, the user s program is essentially a normal ....

Peter Brezany, Michael Gerndt, Piyush Mehrotra, Hans Zima, Concurrent File Operations in a High Performance FORTRAN, Proceedings of Supercomputing '92, pages 230-238, 1992.

....column, these p p processes can each access n= p p data elements, while the other processes access 0 2 We assume that p is a square. 15 system advantages disadvantages nCUBE [16] simple partitioning based on bit permutations all sizes must be powers of 2 array partitioning library [7, 6, 23] supports common array partitioning patterns, high level of abstraction must access full array in one operation nested strided [50] supports the common multidimensional access patterns user needs to compute offsets and strides in all dimensions, must access a full multidimensional ....

....to store array data, the same partitioning scheme can be used. In effect, the distribution of the array data among the processes induces a partitioning of the file segment that stores the array. This has been suggested in a number of libraries, especially in the context of providing I O for HPF [7, 6, 23]. Naturally, it allows all the common partitioning patterns to be expressed. The interface supported by these libraries is a high level interface suitable for direct use by programmers, and using the same abstraction (i.e. partitioned arrays) An analogous low level interface has also been ....

[Article contains additional citation context not shown here]

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima, "Concurrent file operations in a high performance FORTRAN ". In Supercomputing '92, pp. 230--237, Nov 1992.

....83.66 5 Related Work There has been some related research in software support for high performance parallel I O. Vesta is a parallel file system designed and developed at IBM T. J. Watson Research Center [CFPB93, CF94] Language support for out of core data parallel programs are proposed in [BGMZ92, Sni92] File declustering, where different blocks are stored on distinct disks is suggested in [LKB87] This is used in the Bridge File System [DSE88] in Intel s Concurrent File System (CFS) Pie89] and in various RAID schemes [PGK88] The effects of prefetching blocks of a file in a ....

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent File Operations in a High Performance Fortran. In Proceedings of Supercomputing '92, pages 230--238, November 1992.

....5 gives the design and implementation of parallel I O libraries. Finally, Section 6 discusses experimental results, and Section 7 concludes this paper. 2 Related Work The concept of extending the data distribution patterns from memory levels to file levels is first pioneered by Brezany et al.[4]. Their work supports concurrent file operations on Vienna FORTRAN. The system in supporting data distribution for files can be basically classified into two categories: shared storage system and shared nothing environment. Brezany s work is based on a shared storage system, and is supported by ....

....ring and the other being a half ring. The whole algorithm is then carried out by each fragment computing the interactions among each other, stretching the vortexes, and saving the data on every M iterations, for some constant M. The main program loop looks like class Fragment double pos1[m 1][4],vel[m 1] 4] cir[m 1] nber[mnr 1] DistributedArray Fragment Volume( MAXPROC] MAXPROC] Block] main( pfp=pCreate( pfs user local project vortex.iof , w r ) for (iter=1; iter maxiteration; i ) calculate velocity (0) calculate velocity (1) stretch( ....

[Article contains additional citation context not shown here]

Peter Brezany, Michael Gerndt, Piyush Mehrotra, Hans Zima. Concurrent File Operations in a High Performance Fortran, Proceedings of Supercomputing '92, 1992.

....the applications needs for write only files. In addition to full file systems, there are numerous interfaces that are designed to allow programmers to describe their I O needs at a higher semantic level. These interfaces are sometimes tightly integrated into a particular language such as HPF [BGMZ92, HPF93] or CMF [Thi94] There are also many language independent libraries to support parallel I O, usually to support distributed matrices [TBC 94, SW94] The Jovian project explores the issues relating to the storage of irregular structures [BBS 94] Finally, there are also plans to ....

Peter Brezany, Michael Gernt, Piyush Mehotra, and Hans Zima. Concurrent file operations in a High Performance FORTRAN. In Proceedings of Supercomputing '92, pages 230--237, 1992.

....provide no support whatsoever. The inconvenience of having to explicitly specify and control file access for a given data distribution has prompted recent proposals for the inclusion of parallel I O support primitives into parallel programming languages such us HPF Fortran [7] and Vienna Fortran [2]. These primitives could then be implemented as library routines which accept a description of the desired data distribution from the user, and manage data access based upon the correspondence between the user mapping, and the mapping defined by the parallel file system (i.e. the distribution of ....

....that permit the expression of mappings from the problem domain to the processing domain, allow a user to decompose, distribute and align arrays in the most appropriate fashion for the underlying computation. An example of parallel languages which support data distribution includes Vienna Fortran [2], Fortran D [5] and High Performance Fortran or (HPF) 7, 6] In order to address the I O bottleneck problem, these languages propose to provide some support for parallel I O operations. Important examples include Vienna Fortran [2] and High Performance Fortran [7, 6] 3 Analysis of Data ....

[Article contains additional citation context not shown here]

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent File Operations in a High Performance Fortran. Supercomputing'92, pages 230238, November 1992.

....I O. Paleczny et al. 89] are also developing techniques for compiling out of core data parallel programs. Brezany et al. 18] are working on compilation techniques for out of core programs in the context of Vienna Fortran. Language extensions for out of core data parallel programs are proposed in [8, 17, 109, 18]. There has been a lot of effort to improve parallel I O performance both by hardware and software means. Various RAID schemes (Redundant Arrays of Inexpensive CHAPTER 1. INTRODUCTION 11 Disks) are proposed in [91] for better performance, reliability, power consumption and scalability. An ....

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent File Operations in a High Performance Fortran. In Proceedings of Supercomputing '92, pages 230--238, November 1992.

....High Performance Fortran provide explicit directives to describe distribution of data on processors. However, no directives are provided to specify data distribution on disks. Various attempts have been made to provide compiler directives for describing data distribution on disks (Vienna Fortran [BGMZ92, ZBC 92] HPF [Sni92] However at present there is no consensus about how the distribution information should be passed to the compiler. There exist several problems in defining such directives. The out of core arrays used in the programs are stored as files. Hence in case of out of core ....

.... [Iri88] An excellent description of compiler transformations is given in [BGS93] Wolf and Lam propose an elegant loop transformation theory to improve locality and parallelism [WL91, Wol92] Language extensions for out of core data parallel programs are proposed by the Vienna Fortran group [BGMZ92] Marc Snir of IBM has submitted a proposal [Sni92] for I O in HPF to the HPF Forum. 10 Conclusions PASSION provides software support for high performance parallel I O on distributed memory parallel computers. It provides support for compiling out of core data parallel programs, parallel ....

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent File Operations in a High Performance Fortran. In Proceedings of Supercomputing '92, pages 230--238, November 1992.

....and other idiosyncrasies of the machine. In order that these languages can be used for programming Grand Challenge Applications, it is essential that the compiler can automatically translate out of core data parallel programs. Language support for out of core programs has been proposed in [3, 8, 20]. We propose a framework by which a compiler together with appropriate runtime support can translate an out of core HPF program to a message passing node program with explicit parallel I O. Although we use HPF as the source language, the translation technique is applicable to any other data ....

Brezany, P., Gerndt, M., Mehrotra, P., and Zima, H. Concurrent File Operations in a High Performance Fortran. In Proceedings of Supercomputing '92 (November 1992), pp. 230--238.

....advantages offered by its low level nature. 8.3 Higher level Interfaces There are numerous interfaces that are designed to allow programmers to describe their I O needs at a higher semantic level. These interfaces are sometimes tightly integrated into a particular language such as HPF [Lov93, BGMZ92, HPF93] or CMF [Thi94] There has also been great deal of research in designing libraries to support parallel I O. The focus of much of this research has been the support of distributed matrices, particularly for data parallel variants of Fortran [CBH 94, TBC 94, TG96] Other libraries ....

Peter Brezany, Michael Gernt, Piyush Mehotra, and Hans Zima. Concurrent file operations in a High Performance FORTRAN. In Proceedings of Supercomputing '92, pages 230--237, 1992.

....for multifiles , which are too closely tied to the number of physical processors being employed [2, 7] Virtual file pointers also maximize asynchrony, which is very important for efficiency on MIMD hardware. Systems that allow i o operations on parallel aggregates, but with a single file pointer [1, 9], force over synchronization of many applications. The C programming language [3] is centered upon the notion of a shape. A shape is a template to which parallel data values are aligned. In fact, C shapes are very similar in spirit to High Performance Fortran templates. A shape is ....

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent file operations in a high performance FORTRAN. Technical Report 92--46, Institute for Computer Applications in Science and Engineering, NASA Langley Research Center, 1992.

....of processors) Following the current language semantics, accesses in most cases will have to be sequentialized. Various proposals for distributing files over disks are summarized in the HPF Journal of Development [54] Other work in striping files and distributing over disks is discussed in [15, 21, 27, 22, 57] 2.3.3 Out of Core and Persistent Arrays Many potential applications of HPF operate on large quantities of data. Primary data structures for these applications reside on disks. These data structures are termed out of core. Specifying outof core data structures within HPF means that application ....

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent File Operations in a High Performance Fortran. In Proc. Supercomputing '92, pages 230--238, November 1992.

No context found.

Brezany P., Gerndt M., Mehrotra P., Zima H., Concurrent File Operations in a High Performance Fortran. In Proceedings of Supercomputing'92, (November 1992), 230--238.

....the distribution of data impacts performance. So far, the existing HPF programming environments do not provide an appropriate framework for programming and implementing I O intensive engineering and scientific applications. Parallel I O support primitives have been included into Vienna Fortran [2]. Later, similar constructs have been proposed for HPF [1,17] However, they are not implemented by now in any production compiler. Most current multiprocessor file systems are primarily derivates of Unix file systems both in their interface and their implementation. A parallel file system ....

P. Brezany, M. Gerndt, P. Mehrotra, H. Zima, Concurrent file operations in a High Performance FORTRAN, in: Proc. Supercomputing '92, 230--23.

....operations can be classified into three groups: data transfer, inquiry and file manipulation. They can be used by the programmer to provide information which will allow the compiler and runtime environment to optimize the transfer of data to and from secondary storage. Details can be found in [5, 43]. 5 Vienna Fortran and HPF Recently, a consortium of researchers from industry, government labs and academia formed the High Performance Fortran Forum to develop a standard set of extensions for Fortran 90 which would provide a portable interface to a wide variety of parallel architectures. The ....

....which own them. 6.3 Implementation In the Vienna Fortran Compiler, implementation of the above inspector executor strategy is based on the Parti routines ( 38, 41] The compiler automatically inserts them into the code when compiling programs with forall loops. Details can be found in [5]. Parti routines represent a concrete system that addresses the specific problems related to the fact that either distributions of data or data dependence patterns are determined by program data which is not known until run time. A library of primitives have been constructed to support the ....

P.Brezany,M.Gerndt,P.Mehrotra,and H.Zima. Concurrent File Operations in a High-Performance Fortran. In: Proc. Supercomputing'92, Minneapolis, IEEE Computer Society Press, pp.230-237 (November 1992).

....all vendors of the present generation parallel computers. However, the HPF language constructs and associated programming environments do not provide an appropriate framework for programming and implementing I O intensive applications. Parallel I O support primitives have been included into VF [3] and later, similar constructs were proposed for HPF [7, 1] However, they have not been approved by any implementation by now. Any implementation of these constructs is strongly dependent on the runtime system, which supports parallel I O. Until now, there were considered two ways for providing ....

....The design and the development of this system is determined by the following characteristics: parallelism, efficient data administration, abstract I O model and scalability. The rest of this paper is organized as follows: Section 2 outlines our approach that partially builds on the work [3, 6]; the language support for parallel I O is presented in Section 3; the basic compilation method and advanced optimizations are discussed in Section 4; Section 5 presents the basic features of VIPIOS and Section 6 concludes with a brief summary. 2 Our Approach In typical computational science ....

Brezany P., Gerndt M., Mehrotra P., Zima H., Concurrent File Operations in a High Performance Fortran. In Proceedings of Supercomputing'92, (November 1992), 230--238.

No context found.

Peter Brezany, Michael Gernt, Piyush Mehotra, and Hans Zima. Concurrent file operations in a High Performance FORTRAN. In Proceedings of Supercomputing '92, pages 230--237, 1992. 99

No context found.

P. Brezany, M. Gerndt, P. Mehrotra, and H. Zima. Concurrent file operations in a high performance FORTRAN. In Proceedings of Supercomputing '92, Minneapolis, MN, November 1992.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC