Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, 1989. 18  Home/Search   Document Not in Database   Summary   Related Articles   Check

....file system for IBM s SP2, allows users and applications to interact with it exactly as they would interact with AIX file system [20] Indeed, the PIOFS is mounted on each of the nodes of the SP 2 using AIX s standard Virtual File System interface. Bridge was one of the earliest file systems [21], and is unusual in not separating I O nodes from compute nodes. A file system interface proposed for the nCUBE is based on a two step mapping of a file into the compute node memory [22] Galley is another parallel file system for IBM SP2 and it is based on a new three dimensional structuring of ....

Peter C Dibble. A Parallel Interleaved File System. Ph.D. Thesis, University of Rochester, March 1990.

....incomplete block designs, and contrast it to a layout proposed by Merchant and Yu [Menon92a] which supports more combinations of array size (C) and number of units per parity stripe (G) in large arrays, at the cost of higher complexity. 3.2.1. Layout goodness criteria Extending from prior studies [Lee90, Dibble90, Reddy91, Merchant92a], we have identified six criteria for a good disk array layout. 1. Single failure correcting. No two units (whether data or parity) contained in the same parity stripe may reside on the same physical disk. This is the basic characteristic of any failure tolerating organization. In arrays in which ....

P. Dibble, "A Parallel Interleaved File System," University of Rochester Technical Report 334, 1990.

....them with great difficulty. Programmers need a higher level interface to easily take advantage of parallel I O. 5 Existing multiprocessor file system interfaces Several researchers have discussed parallel I O interfaces for MIMD multiprocessors. Dibble, in his design of the Bridge file system [8], defines three interfaces: standard, which is essentially our conventional interface; parallel open, in which a control process issues all the read and write requests, automatically transferring one record in or out of every process; and tools. Tools have access to the local file systems of ....

....position, and does not directly specify a mapping from process to position, as in the nCUBE mappings. A given file position may be mapped by any number of processes (including zero) Also note that self scheduled access, through a global file pointer, is still possible. Logical Records. Dibble [8] argues for direct support for logical records in the file system. The Unix file system does not have any built in support for logical records, in contrast to some traditional systems (typified by commercial mainframes) Such support increases the complexity of the file system, but there are good ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....file systems. Section 3 describes some simulations and their results and Section 4 describes some measurements on a real system. We summarize our conclusions in Section 5. 2 Background There are many different parallel file systems [Kri94, Pie89, FPD93, Roy93, LIN 93, DdR92, CF94, Dib90, DSE88, MS94, HdC95, HER 95] Most, though not all, are designed for machines that have dedicated I O nodes. Most are based on a fairly traditional Unix like interface, in which individual processes make a request to the file system for each piece of the file they read or write. ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....them with great difficulty. Programmers need a higherlevel interface to easily take advantage of parallel I O. 5 Existing Multiprocessor File System Interfaces Several researchers have discussed parallel I O interfaces for MIMD multiprocessors. Dibble, in his design of the Bridge file system [Dib90] defines three interfaces: standard, which is essentially our conventional interface; parallel open, in which a control process issues all the read and write requests, automatically transferring one record in or out of every process; and tools. Tools have access to the local file systems of each ....

....position, and does not directly specify a mapping from process to position, as in the nCUBE mappings. A given file position may be mapped by any number of processes (including zero) Also note that self scheduled access, through a global file pointer, is still possible. Logical Records. Dibble [Dib90] argues for direct support for logical records in the file system. The Unix file system does not have any built in support for logical records, in contrast to some traditional systems (typified by commercial mainframes) Such support increases the complexity of the file system, but there are good ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....which determine how a file pointer is shared among clients running in individual nodes [7, 4, 18] The HFS [20] and KSR1 [17] file systems use a memory mapped interface. The nCUBE [9] and Vesta [5] allow more user control over data layout by providing per process logical views of the data. In PIFS [11], the file system controls which processor handles which part of the file to exploit memory locality. Intel s CFS [14, 22] on the other hand, provides a Unixlike interface with a choice of four I O modes to coordinate parallel access. 2.3 The CM 5 The CM 5 is a scalable message passing ....

P. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....is shared among clients running on individual nodes [7, 4, 18, 14, 22] The Hurricane [20] and KSR1 [17] file systems use a memorymapped interface. The nCUBE [9] and Vesta [5] file systems allow more user control over data layout by providing per process logical views of the data. In PIFS (Bridge) [11], the file system controls which processor handles which part of the file to exploit memory locality. 2.3 The CM 5 The CM 5 is a scalable message passing multiprocessor. It may contain from tens to thousands of processing nodes (PNs) and a few Control Processors (CPs) Each PN has only private ....

P. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....I O in three other situations: data dependent distribution (Section 3) data dependent filtering (Section 4) and working with irregular subsets (Section 5) We present conclusions and a look toward the future in Section 6. 2 Background There are many different parallel file systems [DSE88, Pie89, Dib90, DdR92, Mas92, FPD93, LIN 93, Roy93, CF94, Kri94, MS94, HdC95, HER 95] Most are based on a fairly traditional Unix like interface, in which individual processes make a request to the file system for each piece of the file they read or write. Increasingly common, however, are specialized ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....files may be interleaved over the disks, but the multiple controllers and independent access to the disks make this technique different from disk striping. Examples of this I O architecture include the Concurrent File System [15, 6] for the Intel iPSC 2 multiprocessor, and the Bridge file system [4, 3] for the BBN Butterfly multiprocessor. Caching commonly used disk blocks can significantly improve file system performance [20] and indeed is a technique used in most modern file systems. Prefetching is also successful in uniprocessor file systems [20, 18, 19, 17] The central idea behind ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....contiguous chunk of the file. The file system is thus faced with concurrent small requests from many processors, instead of the single large request that would have occurred on a uniprocessor. Indeed, since most multiprocessor file systems [CF94, FPD93, Pie89, Roy93, DdR92, LIN 93, BGST93, Dib90, DSE88] decluster file data across many disks, each application request may be broken into even smaller requests that are sent to different IOPs. It is difficult for the file system, which is distributed across many I O processors, to recognize these requests as a single coordinated request, and ....

....ra 3 3 3 3 3 3 3 TC rn TC rb 2 2 2 2 2 2 2 TC rc Theta Theta Theta Theta Theta Theta Theta Figure 8: Similar to Figure 7, but here all cases used the random blocks disk layout. 7 Related work Disk directed I O is somewhat reminiscent of the PIFS (Bridge) tools interface [Dib90] in that the data flow is controlled by the file system rather by than the application. PIFS focuses on managing where data flows (for memory locality) whereas disk directed I O focuses more on when data flows (for better disk and cache performance) Some parallel database machines use an ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....records would never be sent to CPs, reducing network traffic (for experimental results, see [Kot95a] In systems with slow or congested networks, that lower traffic would lead to better throughput. 9 Related work Disk directed I O is somewhat reminiscent of the PIFS (Bridge) tools interface [Dib90] in that the data flow is controlled by the file system rather than by the application. PIFS focuses on managing where data flows (for memory locality) whereas disk directed I O focuses more on when data flows (for better disk and cache performance) Some parallel database machines use an ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....one of the earliest parallel file systems, has disks on every node their model does not distinguish between CPs and IOPs. ridge provides both a traditional Unix like interface, and a more complex interface that allows applications to explicitly access the local file systems on each node [Dib90] Intel s Concurrent File System (CFS) Pie89, Nit92] frequently cited as the canonical first generation parallel file system, and its successor, PFS, are examples of file systems that provide a linear file model to the applications, and offer a Unix like interface to the data. Other examples of ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....year. So, the access time gap is widening every year, increasing a problem that has been defined as the I O crisis [13] One solution to the I O crisis is to use a parallel I O subsystem, to connect many disks to the computer in parallel, and to spread individual files across all disks ( 13] 5] [7]) The utilization of parallelism within the I O subsystem is the only way to overcome such limitation, avoiding the restrictions of system performance due to overloaded I O resources. A parallel I O system should include both the parallel execution of several independent I O transfers and the ....

Dibble P.C., A Parallel Interleaved File System, PhD. Thesis, University of Rochester, March, 1990.

....separate compute processors (CP) and I O processors (IOP) concurrent small requests from many processors, instead of the single large request that would have occurred on a uniprocessor. Indeed, since most multiprocessor file systems [CF94, FPD93, Pie89, Roy93, DdR92, LIN 93, BGST93, Kri94, Dib90, DSE88, FBD94] decluster file data across many disks, each application request may be broken into even smaller requests that are sent to different IOPs. It is difficult for the file system, which is distributed across many I O processors, to recognize these requests as a single coordinated ....

....random,128 b) LU time: DDIO TC with 1 KB blocks 0 20 40 60 80 100 contig,16 contig,32 contig,128 random,16 random,32 random,128 Figure 16: Just like Figure 14, but using a 1 KB block size. 8 Related work Disk directed I O is somewhat reminiscent of the PIFS (Bridge) tools interface [Dib90] in that the data flow is controlled by the file system rather than by the application. PIFS focuses on managing where data flows (for memory locality) whereas disk directed I O focuses more on when data flows (for better disk and cache performance) Some parallel database machines use an ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....with separate compute processors (CP) and I O processors (IOP) concurrent small requests from many processors, instead of the single large request that would have occurred on a uniprocessor. Indeed, since most multiprocessor file systems [CF94, FPD93, Pie89, Roy93, DdR92, LIN 93, BGST93, Dib90, DSE88] decluster file data across many disks, each application request may be broken into even smaller requests that are sent to different IOPs. It is difficult for the file system, which is distributed across many I O processors, to recognize these requests as a single coordinated request, and ....

....caching varied, disk directed I O consistently provided excellent performance, at least as good as traditional caching, often independent of access pattern, and often close to hardware limits. 7 Related work Disk directed I O is somewhat reminiscent of the PIFS (Bridge) tools interface [Dib90] in that the data flow is controlled by the file system rather by than the application. PIFS focuses on managing where data flows (for memory locality) whereas disk directed I O focuses more on when data flows (for better disk and cache performance) Some parallel database machines use an ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....parity based on balanced incomplete block designs, and contrast it to a layout proposed by Merchant and Yu [Merchant92] which supports more configurations of large arrays, at the cost of higher complexity. 4.1. Layout goodness criteria Extending from non declustered disk array layout research [Lee90, Dibble90], we have identified six criteria for a good disk array layout. 1. Single failure correcting. No two stripe units in the same parity stripe may reside on the same physical disk. This is the basic characteristic of any single failure tolerating redundancy organization. In arrays in which groups of ....

P. Dibble, "A Parallel Interleaved File System," University of Rochester Technical Report 334, 1990.

....is doubled every year. So, the access time gap is widening every year. This problem has been defined as the I O crisis [6] One solution to the I O crisis is a parallel I O subsystem, which may connect many disks to the computer in parallel, spreading individual files across all disks ( 6] 1] [2]) The utilization of parallelism within the I O subsystem is the only way to overcome such limitation avoiding the restrictions of system performance due to overloaded I O resources. However, parallel disk hardware is not the complete answer to the I O crisis. It is equally important to design ....

P. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, USA, March 1990.

....has a processor and memory, with a high speed network. Nodes with attached disks are usually reserved as I O nodes, while applications run on some cluster of the remaining compute nodes. In the past few years, many parallel file systems have been described in the literature, including Bridge PFS [Dib90] CFS [Pie89] nCUBE [DdR92] OSF PFS [Roy93] sfs [LIN 93] Vesta PIOFS [CFP 95] HFS [KS96] PIOUS [MS94] RAMA [MK95] PPFS [HER 95] Scotch [GSC 95] and Galley [NK96a, NK96b] Many more techniques for improving the performance of parallel file systems have been described, ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....BGST93, Kot93, Pie89, Roy93] HFS [KS93] and the KSR1 [Ken92] file system, use a memory mapped interface. On the nCUBE [DdR92] and in Vesta [CFPB93] the user has more control over data layout. These systems provide a per process logical view of the data. In PIFS (Parallel Interleaved File System) Dib90] the file system controls which processor handles which part of the file to exploit memory locality. Intel s CFS (Concurrent File System) FPD93, Nit92, Pie89] on the other hand, provides a Unix like interface with a choice of four I O modes to coordinate parallel access. On the CM 5 the I O ....

P. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....can completely overwhelm the processor handling the file system. It is easy to find problems that are I O dominated even on uniprocessors. Copying a file, for instance, involves about ten machine instructions at the application level and about 2000 instructions of system overhead per block [6]. This CPU activity would take 0.1 to 1 millisecond, while a fast disk system would need at least 10 milliseconds to accomplish this operation. A factor of 10 to 100 in I O performance improvement would be needed to allow for the processor to execute the program without any I O penalty. Currently ....

P. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, USA, March 1990.

....data rate. In recent years striping has been applied to file systems as a whole. In these file systems the blocks of each file are striped across multiple storage devices. These storage devices may be disks, as in HPFS [Poston88] I O nodes in a parallel computer, as in CFS [Pierce89] and Bridge [Dibble90], or they may be network file servers as in Swift [Cabrera91] It is important to note that these systems stripe on a per file basis, therefore they work best with large files. Small files are a kind of Catch 22: if they span all the storage devices then the amount stored on each device will be ....

Peter C. Dibble, "A Parallel Interleaved File System", Ph.D. Thesis, University of Rochester, 1990.

....processors. The files may be interleaved over the disks, but the multiple controllers and independent access to the disks make this technique different from disk striping. Examples of this architecture include the Concurrent File System [14, 6] for the Intel iPSC 2 multiprocessor, and the Bridge [4, 3] file system for the BBN Butterfly parallel computer. File caching is a technique used in most modern file systems. Caching has not been studied for parallel file systems, but Alan Smith has extensively studied disk caching in uniprocessors with general purpose workloads. In [16] his simulations ....

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, March 1990.

....we describe our layout goals, and the technique used to achieve them. We then comment on the generality of our approach. 4.1. Layout goals Previous work on declustered parity has left open the problem of allocating parity stripes in an array. Extending from non declustered parity layout research [Lee90, Dibble90 ], we have identified six criteria for a good parity layout. The first four of these deal exclusively with relationships between stripe units and parity stripe membership, while the last two make recommendations for the relationship between user data allocation and parity stripe organization. ....

P. Dibble, "A Parallel Interleaved File System," University of Rochester, Technical Report 334, 1990.

No context found.

Peter C. Dibble. A Parallel Interleaved File System. PhD thesis, University of Rochester, 1989. 18

No context found.

P. C. Dibble. A Parallel Interleaved File System. PhD thesis, Department of Computer Science, University of Rochester, Rochester, NY, Mar. 1990.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC