T.W. Pratt, J.C. French, P.M. Dickens, S.A. Janet, Jr., A comparison of the architecture and performance of two parallel file systems, in: Fourth Conference on Hypercube Concurrent Computers and Applica- tions, 1989, pp. 161-166.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....Disk activity over time for RAMA during a read of a I GB file. The access pattern is the same as the right graph of Fig. 10. E.L. Miller, R.H. Katz Parallel Computing 23 (1997) 419 446 5.3. Interconnection network utilization 5.3.1. Network load distribution Many older parallel file systems [21,23] required data placement hints from programs to reduce network traffic as well as balance disk traffic. On some older machines, each interprocessor link was slower than 10 MB s hardly faster than a disk. On such a system, pseudo random placement as done in RAMA would be a poor choice because it ....

T.W. Pratt, J.C. French, P.M. Dickens, S.A. Janet, Jr., A comparison of the architecture and performance of two parallel file systems, in: Fourth Conference on Hypercube Concurrent Computers and Applica- tions, 1989, pp. 161-166.

....necessarily have to be sent to the disk nodes in exactly the right order. The situation for networks is quite different in that the parallel system must meet the order expected by the outside. An interesting issue is the relative location in the system of nodes supporting disk I O and networks [17]. Figure 5 shows two extreme cases. In the first case, both the disk and network interfaces are part of a single I O space , and it is possible to access files on the disk system without going through the Nodes Compute Network System Network System Figure 5: Organizations of I O space ....

T.W. Pratt, J. C. French, P.M. Dickens, and S. A. Janet Jr. A comparison of the architecture and performance of two parallel file systems. In Proceedings of the Fourth Conference on Hypercubes, Concurrent Computers, and Applications, volume 1, pages 161-166, California, March 1989.

....(slow) storage devices into a faster logical storage service, making all applications unaware of this aggregation. In Swift, sets of storage servers work concurrently to satisfy the requests made by clients. Several concurrent I O architectures, such as Imprimis ArrayMaster [4] DataVault [5] CFS [6], RADD [7] and RAID [3, 8] are based on this observation. Mainframes [9, 10] and super computers [11] have also exploited this approach. Swift is a client server system made up of independently replaceable components. Clients are connected to sets of storage servers through an interconnection ....

....striping in a distributed environment, striping files over multiple servers in a local area network. Examples of some commercial systems that utilize disk striping include super computers [11] DataVault for the CM 2 [5] the airline reservation system TPF [9] the IBM AS 400 [10] CFS from Intel [6], and the Imprimis ArrayMaster [4] Hewlett Packard is developing a system called DataMesh that uses an array of storage processors connected by a high speed switched network [4] For all of these the maximum data rate is limited by the interconnection medium which is an I O channel. Higher data ....

T. W. Pratt, J. C. French, P. M. Dickens, and S. A. Janet, "A comparison of the architecture and performance of two parallel file systems," in Proceedings of the 4 th Conference on Hypercubes, (Monterey), Mar. 1989. 21

....them in parallel. The principle behind our architecture is simple: aggregate arbitrarily many (slow) storage devices into a faster logical service, making all applications unaware of this aggregation. Several concurrent I O architectures, such as Imprimis ArrayMaster [4] DataVault [5] CFS [6, 7] and Raid [3, 8] are based on this observation. Mainframes [9, 10] and super computers [11] have also exploited this approach. 1 Unix is a trademark of AT T Bell Laboratories 2 Distribution Agent . Storage Agents Storage Mediator Client Interconnection Medium Figure 1: ....

....the first to use disk striping in a distributed environment, striping files over multiple servers. Examples of some commercial systems that utilize disk striping include super computers [11] DataVault for the CM 2 [5] the airline reservation system TPF [9] the IBM AS 400 [10] CFS from Intel [6, 7], and the Imprimis ArrayMaster [4] Hewlett Packard is developing a system called DataMesh that uses an array of storage processors connected by a highspeed switched network [19] For all of these the maximum data rate is limited by the interconnection medium which is an I O channel. Higher ....

T. W. Pratt, J. C. French, P. M. Dickens, and S. A. Janet, "A comparison of the architecture and performance of two parallel file systems," in Proceedings of the 4 th Conference on Hypercubes, (Monterey), Mar. 1989.

....disks to separate processors or ports on the interconnection network, as shown in Figure 1. In either case files are declustered over many disks. We call the latter structure Parallel Independent Disks (PID) Examples of multiprocessors using a PID architecture include the Intel [14, 15] nCUBE [22, 5, 27], and Kendall Square Research [20] multiprocessors. Network Memory Processor Memory Processor Memory Processor Disk Disk Disk Figure 1: Parallel Independent Disks (PID) in an MIMD multiprocessor. 3 The workload Parallel file systems and the applications that use them are not sufficiently mature ....

....forcing the programmer to specify the set of disks, disk files, or disk blocks, then transparency is lost and the interface is much harder to use. An example of this situation is in [3] Another example is the nCUBE file system prior to 1992, which does not distribute a single file across disks [27]. We believe that it is important to have a single name (e.g. Unix pathname) that defines the parallel file, and to leave the rest to the file system. 4.4 Segmented files Consider programming the read only segmented access pattern. In this pattern, the file is divided into disjoint segments, ....

[Article contains additional citation context not shown here]

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....benchmark kernel [12] the Input Output (I O) performance is assessed. Each client will write to disk 12.5 MBytes of data which is subsequently read twice; once in a sequential fashion and once in a random fashion. Goal: Evaluation of scalability of I O subsystem. creat and fill. This benchmark [13] generates a single large output file (10MBytes) by using multiple client processes. Each client is assigned a disjoint partition of the output file to fill. Conventional files and multifiles are used. Goal: Evaluation of performance of writing operations on a shared file. segmented read. In this ....

T. Pratt, J. French, P. Dickens, and S. Janet. A Comparison of the Architecture and Performance of Two Parallel File Systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166. Golden Gate Ent., 1989. UPM 88 Confidential

....processors or ports on the interconnection network, as shown in Figure 2. In either case files are declustered over many disks. We call the latter structure Parallel Independent Disks (PID) Examples of multiprocessors using a PID architecture include the Intel [Int88, Int91] nCUBE [nC90, DdR92, PFDJ89] and Kendall Square Research [KSR92] multiprocessors. 3 The Workload Parallel file systems and the applications that use them are not sufficiently mature for us to know what access patterns might be typical. Here we define our expectations for parallel file access patterns in a scientific ....

....the programmer to specify the set of disks, disk files, or disk blocks, then transparency is lost and the interface is much harder to use. An example of this situation is in [Cro88] Another example is the nCUBE file system prior to 1992, which does not distribute a single file across disks [PFDJ89] We believe that it is important to have a single name (e.g. Unix pathname) that defines the parallel file, and to leave the rest to the file system. 4.4 Segmented files Consider programming the read only segmented access pattern. In this pattern, the file is divided into disjoint segments, ....

[Article contains additional citation context not shown here]

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161-- 166, 1989.

....1992] File System Organization: The file system built on top of a multiple disk I O subsystem can either provide a single global view or allow the user to access the local disks independently. The Intel CFS for example, makes the underlying multiplicity of disks transparent to the user [Pratt et al. 1989]. The user sees only a uniform logical namespace, and has no way to access the individual disks. In contrast, the nCUBE s file system is composed of one global layer overlaid on top of multiple local file systems [Pratt et al. 1989] The user can directly interact with the local file systems, ....

....makes the underlying multiplicity of disks transparent to the user [Pratt et al. 1989] The user sees only a uniform logical namespace, and has no way to access the individual disks. In contrast, the nCUBE s file system is composed of one global layer overlaid on top of multiple local file systems [Pratt et al. 1989]. The user can directly interact with the local file systems, bypassing the global layer altogether. A similar organization is also used in the Bridge file system [Dibble et al. 1988] In conventional file systems, the fragmentation of a file s data on the disk leads to expensive seek operations ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Proceedings of the Fourth Conference on Hypercubes, Concurrent Computers and Applications, pages 161--166, 1989.

....while some of the authors were in residence at ICASE, Mail Stop 132C, NASA Langley Research Center, Hampton, VA 23681. ily scaled up by increasing the number of processors. The I O bottleneck has been somewhat alleviated in these systems by powerful Concurrent Input Output Systems (CIOSs) [1, 2, 3, 9, 16]) Hardware and software architectures of CIOSs provided by various DMMP vendors differ substantially. For example, the Concurrent File System TM (CFS) developed by Intel for the iPSC 2 and iPSC 860 supercomputers [15] is based on an architecture which is straightforward to use while delivering ....

T.W. Pratt, J.C. French, P.M. Dickens, S.A. Janet, Jr. : A Comparison of the Architecture and Performance of Two Parallel File Systems, Proceedings of the DMCC 4, 161--166, 1989

....is simple: use a high speed interconnection medium to aggregate arbitrarily many (slow) storage devices into a faster logical storage service, making all applications unaware of this aggregation. Several concurrent I O architectures, such as Imprimis ArrayMaster [4] DataVault [5] CFS [6,7], RADD [8] and RAID [3,9] are based on this observation. Mainframes [10,11] and super computers [12] have also exploited this approach. Swift is a client server distributed architecture made up of independently replaceable components. The advantage of this modular approach is that any component ....

....the first to use disk striping in a distributed environment, striping files over multiple servers. Examples of some commercial systems that utilize disk striping include super computers [12] DataVault for the CM 2 [5] the airline reservation system TPF [10] the IBM AS 400 [11] CFS from Intel [6,7], and the Imprimis ArrayMaster [4] Hewlett Packard is developing a system called DataMesh that uses an array of storage processors connected by a high speed network [23] For all of these the maximum data rate is limited by the interconnection medium which is an I O channel. Higher data rates can ....

T. W. Pratt, J. C. French, P. M. Dickens, and S. A. Janet, "A comparison of the architecture and performance of two parallel file systems," in Proceedings of the 4 th Conference on Hypercubes, (Monterey), Mar. 1989.

.... primarily scientific applications show that many of these assumptions are incorrect [KN94, PEK 95, NKP 95] Specifically, it was commonly believed that parallel, scientific applications would have behavior similar to that of existing sequential and vector scientific applications [Pie89, PFDJ89, LIN 93] These applications tend to access large files in large, consecutive chunks [MK91, PP93] Studies of Copyright c fl1996 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....nCUBE, and the Meiko CS 2 are based on this model. 2.2 Caching There have been several file caching studies in multiprocessors. Pratt and French studied the Intel Concurrent File System (CFS) and found that its caching improved performance, but they were not able to compare several policies [PFDJ89, FPD91] More recent studies found that, in general, despite its cache performance, CFS performance lags well behind hardware capabilities [Nit92, BCR93] Miller and Katz [MK91] used application traces from a Cray supercomputer to drive cache simulations, and found that there was not enough ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....algorithms effectively perform their own caching. Existing systems Here we survey some existing systems and their support for the above capabilities. Table 1 summarizes these systems. One of the first commercial multiprocessor file systems is the Concurrent File System (CFS) Pie89, FPD93, PFDJ89] for the Intel iPSC and Touchstone Delta multiprocessors [Int88] CFS declusters files across several I O processors, each with one or more disks. It provides the user with several different access modes, allowing different ways of sharing a common file pointer. Unfortunately, caching and ....

....in the KSR 1. RAID level three organization limits the disk array at each I O node to only fully striped I O. The apparent number of independent disks, therefore, is only the number of I O nodes, rather than the larger number of physical disks. The first file system for the nCUBE multiprocessor [PFDJ89] gives plenty of control to the user. In fact, the operating system treats each disk as a separate file system and does not decluster individual files across disks. Thus, the nCUBE provides the low level access one needs, but no higher level access. The current nCUBE file system [dBC93] supports ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....credit is permitted. the development of most parallel file systems were incorrect [KN94, NK95, PEK 95] It was commonly believed that parallel scientific applications would behave like sequential and vector scientific applications: accessing large files in large, consecutive chunks [Pie89, PFDJ89, LIN 93, MK91] Studies of two parallel file system workloads, running a variety of applications in a variety of scientific domains, at two sites on two architectures, under both data parallel and control parallel programming models, show that many applications make many small, regular, but ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....are associated with selected multicomputer nodes, and are typically of the same complexity as the processing elements (PE) used in the computing nodes. These I O nodes provide access to I O devices such as RAID arrays through standard interfaces such as SCSI and HIPPI, or through a disk controller [28]. They can also function as device file servers, and use part of their local memory as a disk cache. A good comparative description of the I O nodes used in the Intel iPSC 2 and NCUBE 9 multicomputers can be found in [28] The performance of an I O subsystem using multiple I O nodes is strongly ....

....standard interfaces such as SCSI and HIPPI, or through a disk controller [28] They can also function as device file servers, and use part of their local memory as a disk cache. A good comparative description of the I O nodes used in the Intel iPSC 2 and NCUBE 9 multicomputers can be found in [28]. The performance of an I O subsystem using multiple I O nodes is strongly influenced by the schemes used to store, maintain and access data. Using disk striping, one can interleave data across independent disks under the control of a single file system [34] This technique is able to provide ....

T.W. Pratt, J.C. French, P.M. Dickens, and S.A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Proceedings of the Fourth Conference on Hypercubes, Concurrent Computers, and Applications, pages 161--166, March 1989.

....asked to support. For example, it was generally assumed that scientific applications designed to run on a multiprocessor would behave in the same fashion as scientific applications designed to run on sequential and vector supercomputers: accessing large files in large, consecutive chunks [Pie89, PFDJ89, LIN 93, MK91] Instead, our observations show that many scientific applications make many small, regular, but non consecutive requests to the file system. Using the results from our workload characterizations and from performance evaluations of existing multiprocessor file systems, we have ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....Per disk Raw Disk of raw Reference 1 iPSC 860 10 10 8.0 0.80 1 80 [KN93] 2 iPSC 2 4 4 3 0.75 1 75 [FPD93] 3 iPSC 2 16 8 5.54 0.69 1 69 [AS89] 4 iPSC 10 10 5.5 0.55 1 55 [Dun91] 5 iPSC 2 16 8 6.0 0.38 1 38 [Pie89] 6 iPSC 2 4 2 2.3 0.58 1.875 31 [Are91] 7 iPSC 2 4 4 0.62 0. 15 1 15 [PFDJ89] 8 iPSC 860 10 10 0.64 0.064 1 6.4 [Fin93] 9 Delta 64 32 10 0.16 2.5 0.6 [BCR92] 10 Delta 64 32 7.65 0.12 2.5 0.4 [dBC93] nCUBE Model Disks I O nodes Total Per disk Raw Disk of raw Reference 11 nCUBE 2 4 4 4.96 1.24 [DdR92] 12 nCUBE 2 6 6 6.3 1.05 [dR92] 13 nCUBE 2 8 8 3.91 0.49 [dBC93] ....

....10 0.16 2.5 0.6 [BCR92] 10 Delta 64 32 7.65 0.12 2.5 0.4 [dBC93] nCUBE Model Disks I O nodes Total Per disk Raw Disk of raw Reference 11 nCUBE 2 4 4 4.96 1.24 [DdR92] 12 nCUBE 2 6 6 6.3 1.05 [dR92] 13 nCUBE 2 8 8 3.91 0.49 [dBC93] 14 nCUBE 10 8 8 2.5 0.31 [BN90] 15 nCUBE 10 8 8 0.27 0. 03 [PFDJ89] TMC CM 2 DataVault Model Disks I O nodes Total Per disk Raw Disk of raw Reference 16 CM 2 32 (1) 25 0.78 1 78 [KN93] 17 CM 2 32 (1) 4.3 0.13 1 13 [Fin93] TMC CM 5 Scalable File System Model Disks I O nodes Total Per disk Raw Disk of raw Reference 18 CM 5 32 4 54.4 1.7 2 85 [LIN ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....second, which also declusters data over many disks, is to attach independent controllers and disks to separate processors or ports on the interconnection network. We call the latter structure Parallel Independent Disks (PID) Examples of a PID architecture include Intel s Concurrent File System [15, 8, 17], the Bridge simulated file system [6, 5] for the BBN Butterfly, and the file system for the nCUBE 2 [12, 4, 17] While caching has not been studied for parallel file systems, Alan Smith has extensively Kotz and Ellis: Caching and Writeback Policies in Parallel File Systems 5 studied caching in ....

....processors or ports on the interconnection network. We call the latter structure Parallel Independent Disks (PID) Examples of a PID architecture include Intel s Concurrent File System [15, 8, 17] the Bridge simulated file system [6, 5] for the BBN Butterfly, and the file system for the nCUBE 2 [12, 4, 17]. While caching has not been studied for parallel file systems, Alan Smith has extensively Kotz and Ellis: Caching and Writeback Policies in Parallel File Systems 5 studied caching in uniprocessors with general purpose workloads [20] Uniprocessor and distributed system file access patterns have ....

T. W. Pratt, J. C. French, P. M. Dickens, and S. A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....not necessarily have to be sent to the disk nodes in exactly the right order. The situation for networks is quite different in that the parallel system must meet the order expected by the outside. An interesting issue is the relative location in the system of nodes supporting disk I O and networks [17]. Figure 5 shows two extreme cases. In the first case, both the disk and network interfaces are part of a single I O space , and it is possible to access files on the disk system without going through the Compute Nodes . Network Disk System Compute Nodes Network Disk System Figure 5: ....

T. W. Pratt, J. C. French, P. M. Dickens, and S. A. Janet Jr. A comparison of the architecture and performance of two parallel file systems. In Proceedings of the Fourth Conference on Hypercubes, Concurrent Computers, and Applications, volume 1, pages 161--166, California, March 1989.

.... [KN94, NK96a, PEK 95] It was generally assumed that scientific applications designed to run on a multiprocessor would behave in the same fashion as scientific applications designed to run on sequential and vector supercomputers: accessing large files in large, consecutive chunks [Pie89, PFDJ89, LIN 93, MK91] Studies of two different multiprocessor file system workloads, This research was funded by NSF under grant number CCR 9404919 and by NASA Ames under agreement numbers NCC 2 849 and NAG 2 936. running a variety of applications in a variety of scientific domains, on two ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

.... file systems were incorrect [12,18,25] It was generally assumed that scientific applications designed to run on a multiprocessor would behave in the same fashion as scientific applications designed to run on sequential and vector supercomputers: accessing large files in large, consecutive chunks [23,24,15,16]. Studies of two different multiprocessor file system workloads, running a variety of applications in a variety of scientific domains, on two architectures, under both data parallel and control parallel programming models, show that many applications make many small, regular, but non consecutive ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....algorithms effectively perform their own caching. Existing systems Here we survey some existing systems and their support for the above capabilities. Table 1 summarizes these systems. One of the first commercial multiprocessor file systems is the Concurrent File System (CFS) Pie89, FPD93, PFDJ89] for the Intel iPSC and Touchstone Delta multiprocessors [Int88] CFS declusters files across several I O processors, each with one or more disks. It provides the user with several different access modes, allowing different ways of sharing a common file pointer. Unfortunately, caching and ....

....physical disks, the local RAID 3 organization limits the disk array at each I O node to only fully striped I O. The apparent number of independent disks, therefore, is only the number of I O nodes, rather than the larger number of physical disks. The first file system for the nCUBE multiprocessor [PFDJ89] gives plenty of control to the user. In fact, the operating system treats each disk as a separate file system and does not decluster individual files across disks. Thus, the nCUBE provides the low level access one needs, but no higher level access. The current nCUBE file system [Dd93, dBC93] ....

Terrence W. Pratt, James C. French, Phillip M. Dickens, and Stanley A. Janet, Jr. A comparison of the architecture and performance of two parallel file systems. In Fourth Conference on Hypercube Concurrent Computers and Applications, pages 161--166, 1989.

....bytes, or larger blocks How should the overall file system be organized [KOTZ90, PIER89] Almost all the research on these design alternatives has been limited to simulation studies. A few preliminary studies that involve measurement of early versions of the Intel CFS are presented in [ASBU89, BRAD89, PIER89, PRAT89]. Our study here is based on the production software provided by Intel for the system. To our knowledge, this study represents the first published performance measurements on the production system. The Intel CFS represents one slice from the range of design choices mentioned above. In the Intel ....

T. W. Pratt, J. C. French, P. M. Dickens and S. A. Janet, "A Comparison of the Architecture and Performance of Two Parallel File Systems", Proc. 4th Conf. on Hypercube Concurrent Computers and Applications, Monterey, CA, Mar. 1989, 161-166.

No context found.

Pratt, T.W., French, J.C., Dickens, P.M., Janet, S.A., Jr., A comparison of the architecture and performance of two parallel file systems, Proc. Fourth Conference on Hypercube Concurrent Computers and Applications, 1989, 161-166.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC