N. Nieuwejaar and D. Kotz. The Galley Parallel File System. Parallel Computing, 23(4), June 1997.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....the objects near to the requesting machines . We believe, SEN architecture is a perfect match for providing such an infrastructure. However, SEN does not deal with deep archival and concurrent update issues. Parallel file systems also try to alleviate the I O performance and scalability problems [64, 38, 78, 73], but are out of the scope of our current research. 2.4. Hierarchical and Distributed Web Caching 9 2.4 Hierarchical and Distributed Web Caching Web caches exploit reference locality [7] and allow shared data access. Hierarchical proxies such as Harvest Squid [33, 94] improve scalability over ....

N. Nieuwejaar and D. Kotz. The Galley parallel file system. In Proceedings of 10th ACM International Conference on Supercomputing, pages 374--381, Philadelphia, PA, 1996. ACM Press.

....with a UNIX like interface. The nCube parallel I O system [DC92] builds mapping functions between processor s views of a file and disks using address bit permutations. The major deficiency of this approach is that all sizes must be powers of two. Files in the Galley Parallel File System [NK97] are composed of one or more subfiles. Each subfile resides on a single disk and contains one or more forks. The underlying parallel structure of the file is hidden from the application. Galley offers a particular interface which allows simple strided, nested strided and nested batched operations. ....

Nils Nieuwejaar and David Kotz. The Galley Parallel File System. Parallel Computing, 23(4), June 1997.

....algorithms in a parallel file system. 1 Introduction The discrepancy between processor and memory speed on one side, and disks, on the other side, has been identified as a major drawback for applications with intensive I O activity. For addressing this problem, some parallel file systems [5, 6, 4, 3, 13, 2] and libraries [15, 11] have employed mechanisms such as striping a file on several independent disks and allowing parallel file access. Another main problems in parallel I O is efficiently handling byte granularity, non contiguous I O. For instance, parallel scientific applications often access ....

....I O. For instance, parallel scientific applications often access the files non contiguously or their contiguous accesses translates into non contiguous disk accesses[12] MPI IO [11] and Vesta[3] allow setting linear views on non contiguous file data, whereas Galley Parallel File System[13] offers the user a nested strided interface. Parallel I O access characterization studies [12, 1, 16, 17] have found the poor match between I O access patterns of applications and physical layout of data on disks as a large source of I O usage inefficiency. First, a poor match can cause ....

[Article contains additional citation context not shown here]

N. Nieuwejaar and D. Kotz. The Galley Parallel File System. Parallel Computing, 23(4), June 1997.

....These file systems provide easy access to distributed resources, but they are not designed for high performance parallel data access required by parallel applications. Another body of work is parallel file systems, including IBM Vesta [9] and PIOFS [10] Intel Paragon [16] HP Exemplar [5] Galley [15] and so on. These parallel file systems, either commercial or experimental, take advantage of parallel I O techniques, caching, prefetching etc to achieve significant performance improvement. The storage resources of these systems, however, are tightly coupled with the compute nodes, so they do ....

N. Nieuwejaar and D. Kotz. The galley parallel file system. In Proceedings of the 10th ACM International Conference on Supercomputing, Philadelphia, PA, May 1996.

....to work from a number of often distinct areas: file systems, programming environments, and database research. In this section, we discuss work from the three areas. 4.4. 1 Parallel file systems High performance parallel file systems are abundant in the literature: PPFS [HER 95] Galley [NK96] Vesta [CF96] Swift [CL91] CFS [Nit92] SFS [LIN 93] and the SIO specification [BBD 94] However, most assume performance homogeneous devices; thus, performance is dictated by the slowest component in the system, and few of them provide a programming environment, leading to simple ....

Nils Nieuwejaar and David Kotz. The Galley parallel file system. In Proceedings of the 10th ACM International Conference on Supercomputing, pages 374--381, Philadelphia, PA, May 1996. ACM Press.

....It is very easy for the user to change parameters directly in the Java window to get other prediction results. 6. Related work The related work can be divided into four groups. One is parallel file systems, including IBM Vesta [10] and PIOFS [11] Intel Paragon [24] HP Exemplar [5] Galley [23], PPFS [19] PIOUS [22] and so on. These parallel file systems, either commercial or experimental, take advantage of parallel I O techniques, caching, prefetching, etc. to achieve significant performance improvements. The storage of these systems usually includes only secondary storage resources ....

N. Nieuwejaar and D. Kotz, The Galley parallel file system, in: Proc. of the 10th ACM International Conference on Supercomputing, Philadelphia, PA (May 1996) pp. 374--381.

.... There have been many performance studies conducted on le systems for message passing multiprocessors, multicomputers, and clusters such as the performance of the Andrew File System [27] Sprite Network File System [47] Thinking Machines Scalable File System [38] Galley Parallel File System [48, 49, 50], IBM Vesta Parallel File System [17] IBM General Parallel File System [25] IBM Parallel I O File System [79] Intel Parallel File System [79] Intel Concurrent File System [10] and Parallel Virtual File System [13] However, there have been few performance studies conducted on parallel le ....

....multiprocessors. Due to the coarser granularity of sharing in message passing multiprocessor le systems, these designs do not apply well to tightly coupled shared memory multiprocessor le systems such as HFS. Kotz and Nieuwejaar designed a le system for multiprocessors called Galley [48, 49, 50]. Galley is a le system targetted for scienti c workloads, and it has been evaluated on message passing systems such as the IBM SP2 MPP. Galley is not speci cally targetted for shared memory processors. Dibble et al. 19] created Bridge as a multi layered le system designed for parallel ....

Nils Nieuwejaar and David Kotz. The Galley Parallel File System. In ACM International Conference on Supercomputing, pages 374-381. ACM Press, May 1996.

....this thread type on every nodes. Second, Control Thread, controls file opening, file closing and handles information related to files. Last, GFP Thread provides atomic operation of read, write. Application interface with SPIFFI through library contains C APIs. It is a superset of Unix I O. Galley [13], a parallel file system, designed to allow application ability to specify data layout to yield better degree of parallelism. Galley supports assumptions that parallel data can be arranged into one of stride access patterns as follows. First, Simple Stride or Singly nestedpattern, patterns ....

Nils Nieuwejaar, David Kotz, "The Galley Parallel File System", Dartmouth College, Department of Computer Science, August 1996. The technical paper can be available at: http://www.cs.dartmouth.edu/-nils/galley.html

....layer, whereas ERAID is designed to allow more functionality to be placed within the file system. Earlier work at HP on DataMesh also proposes more sophisticated interfaces for network attached storage [50] Our informed approach is also similar to a large body of work in parallel file systems [17, 24]. Most parallel file systems expose disk parallelism, but they allow the application itself, and not the file system, to manage it. Better control over redundancy in a parallel file system has also been proposed [9] In that work, the computation of parity is put under user control, and in doing ....

N. Nieuwejaar and D. Kotz. The Galley Parallel File System. In Proceedings of the 10th ACM International Conference on Supercomputing, pages 374--381, Philadelphia, PA, May 1996. ACM Press.

....for each compute process, who use TCP IP for moving data between themselves and clients. Their interface supports strided and batch accesses. There is no collective I O support; presumably this would be implemented at a higher level. All application tasks separately open and close files. In [37] the performance of the system is tested using a simulated disk, giving them the level of control over the disk and file system policies that would only be available inside the kernel. In two separate situations they run into problems with regards to network utilization and eventually note that ....

Nils Nieuwejaar and David Kotz. The Galley parallel file system. Parallel Computing, 23(4):447--476, June 1997.

....level code. The resulting code is very kernel dependent as well, making porting to other operating systems difficult. 3.1. 4 Galley Parallel File System The Galley parallel file system was developed at Dartmouth University and designed to operate on supercomputers and clusters of workstations [36]. The authors assume that some nodes will be used for I O and others for computation, but not both; however, nothing in the 36 design precludes overlap of I O and compute nodes. The focus of Galley s implementation is on low level support for disk directed I O [21] That is, it allows the servers ....

Nils Nieuwejaar and David Kotz. The Galley parallel file system. In Proceedings of the 10th ACM International Conference on Supercomputing, pages 374--381, Philadelphia, PA, May 1996. ACM Press.

....bandwidth I O busses [8] It is generally agreed upon that an optimized I O software can also play a major role in bridging this performance gap. For example, parallel file systems use several optimization techniques such as data striping, caching, and prefetching to reduce hide I O latencies [17, 6, 15]. In order to eliminate the difficulty in using a parallel file system directly, several research groups proposed high level parallel I O libraries and runtime systems that allow programmers to express access patterns of their codes using program level data structures such as rectilinear array ....

N. Nieuwejaar and D. Kotz. The Galley parallel file system. In Proc. the 10th ACM International Conference on Supercomputing, pages 374-- 381, Philadelphia, PA, May 1996. ACM Press.

.... that enable advanced parallel I O techniques like collective I O, prefetching, and data sieving [11 14] A system based on a flexible and powerful low level interface encourages the development of application specific libraries that provide the interface and features that benefit applications [15]. For example, tailoring the prefetching and caching policies to match the application s access patterns can reduce latency and avoid unnecessary data requests [16 18] and matching data distribution policies to the application s access patterns can optimize parallel access to distributed disks ....

....grid application consists of a set of client processes and a set of remote data servers. The traditional approach is for data servers to run system software, perhaps the server side of a parallel file system, and for client processors to run all application specific code. Even Galley [15], which encourages layers of libraries that provide application specific functionality, contains all the application code on the client processors. Since the network is typically the bottleneck of data intensive grid applications, the placement of data processing functionality is critical. A key ....

N. Nieuwejaar, D. Kotz, The Galley parallel file system, Parallel Comput. 23 (4) (1997) 447--476.

....grids, unlike tightly connected systems, have heterogeneous components, unpredictable performance variations, and span multiple administrative domains. In such an environment, centralized control of resources and functionality is not practical or efficient. The Galley parallel file system [14] proposed that the traditional functionality of parallel file systems be separated into two components: a fixed core, standard on all platforms, that encapsulates only primitive abstractions and interfaces, and a set of high level libraries to provide a variety of abstractions and ....

N. Nieuwejaar and D. Kotz. The Galley parallel file system. Parallel Computing, 23(4):447--476, June 1997.

....about the application programmer s interface (API) One intriguing interface is that of the Vesta parallel file system [CBF93, CF94, CFP 95, CF96] which seems to support regular, non overlapping matrix distributions well. The interface for the Galley parallel file system [NK96a, NK96b, KN96, NK97] attempts to be a low level interface that can be a base for other APIs, so it was natural to consider implementing Vesta s API on Galley. We have also implemented the Panda API [Tho96] and a linear file model similar to many older parallel file systems [Nie96] We implemented the Vesta file ....

Nils Nieuwejaar and David Kotz. The Galley parallel file system. Parallel Computing, 23(4):447--476, June 1997.

....as yet no agreement about the application programmer s interface (API) One intriguing interface is that of the Vesta parallel file system [CBF93, CF94, CFP 95, CF96] which seems to support regular, non overlapping matrix distributions well. The interface for the Galley parallel file system [NK96a, NK96b, KN96, NK97] attempts to be a low level interface that can be a base for other APIs, so it was natural to consider implementing Vesta s API on Galley. We have also implemented the Panda API [Tho96] and a linear file model similar to many older parallel file systems [Nie96] We ....

Nils Nieuwejaar and David Kotz. The Galley parallel file system. In Proceedings of the 10th ACM International Conference on Supercomputing, pages 374--381, Philadelphia, PA, May 1996. ACM Press.

....parallel disk I O, turn o# parity (for systems such as RAID [PGK88] and turn o# caching and prefetching. Until very recently, most current operating systems did not provide the programmer with these capabilities. There has been a recent surge of activity in parallel file systems [CF96, KS97, NK97, MK97, HER 95] and parallel I O interfaces [MPI97, CPD 96] that address some of these requirements. One common feature of many parallel file interfaces is that the programmer to specify the access pattern for each file. Typically, this access pattern is specified with one or more parameters ....

....that determine the mapping of file data to disks, and file accesses to file data. HFS takes a di#erent approach, allowing the programmer to construct a file abstraction from object oriented components, such as a distribution component and a replication component [KS97] The Galley file system [NK97] and the SIO interface [CPD 96] choose a lowlevel philosophy, in which the file system provides only a low level interface with highly flexible primitives. Other interfaces and abstractions, such as MPI 2, Vesta, Panda, or HFS, can be built on top of these lower level interfaces. Some believe ....

Nils Nieuwejaar and David Kotz. The Galley parallel file system. Parallel Computing, 23(4):447--476, June 1997.

No context found.

N. Nieuwejaar and D. Kotz. The Galley Parallel File System. Parallel Computing, 23(4), June 1997.

No context found.

Nils Nieuwejaar and David Kotz. The Galley Parallel File System. Parallel Computing, 23(4), June 1997.

No context found.

N. Nieuwejaar and D. Kotz. The Galley parallel file system. In Proceedings of the 10th International Conference on Supercomputing, pages 374--381, Aug. 1996.

No context found.

Nils Nieuwejaar and David Kotz. The Galley Parallel File System. In ACM International Conference on Supercomputing, pages 374--381. ACM Press, May 1996.

No context found.

N. Nieuwejaar and D. Kotz. The Galley parallel file system. In Proceedings of 10th ACM International Conference on Supercomputing, pages 374--381, Philadelphia, PA, 1996. ACM Press.

No context found.

N. Nieuwejaar and D. Kotz. The Galley parallel file system. In Proceedings of 10th ACM International Conference on Supercomputing, pages 374--381, Philadelphia, PA, 1996. ACM Press.

No context found.

N. Nieuwejaar and D. Kotz. The Galley parallel file system. In Proceedings of 10th ACM International Conference on Supercomputing, pages 374--381, Philadelphia, PA, 1996. ACM Press.

No context found.

N. Nieuwejaar and D. Kotz, "The Galley parallel file system. " Parallel Comp., 23, 447-476 (1997).

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC