D. Kotz. Disk-directed I/O for MIMD Multiprocessors. In Proc. of the First USENIX Symp. on Operating Systems Design and Implementation, 1994.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....Introduction Data intensive applications have presented challenging problems to computational scientists. A major problem is the I O performance when many datasets are stored and frequently accessed. Over the years, scientists have developed techniques for I O optimizations such as collective I O [14, 38, 23, 10], prefetching [15, 13] data sieving [38, 10] caching [8] and so on. But the performance problem still persists because data intensive applications are Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL 60208, choudhar ece.nwu.edu generating more data and ....

D. Kotz. Disk-directed i/o for mimd multiprocessors. In Proc. the 1994.

....and complex scientific data. The user program can send query requests to MDMS to obtain information about data structures that will be accessed. Then, the user can use this information in accessing the HSS in an optimal manner, taking advantage of powerful I O optimizations like collective I O [35, 8, 23], prefetching [19] prestaging [14] and so on. The user program can also send access pattern hints to the MDMS and let the MDMS to decide the best I O strategy considering the storage layout of the data in question. These access pattern hints span a wide spectrum that contains inter processors ....

D. Kotz. Disk-directed i/o for mimd multiprocessors. In Proc. the 1994.

....certain optimizations. Another advantage is that if any processor needs to access some data which was previously modified by some other processor, it can be done using just a read call without any additional synchronization. The idea of collective I O has also been used in other schemes such as in [1, 11, 10]. In the next section, we describe the Extended Two Phase Method for reading sections of out of core arrays. The method for writing sections is analogous and is discussed in Section 6. 5 Reading Sections of Out of Core Array Let us assume that each processor needs to read some regular section ....

....proposed in this paper provides similar functionality using only a single routine for reading sections and a single routine for writing sections, which can be implemented very easily. It does not incur the overhead of managing many processes and the associated context switching. Disk directed I O [11] is another technique which uses collective I O. In disk directed I O, compute processors collectively send a single request to all I O processors, which then perform I O efficiently, and send the data to the compute processors. In other words, it is the I O processors that decide the best way of ....

D. Kotz. Disk-directed I/O for MIMD Multiprocessors. In Proceedings of the First Symposium on Operating Systems Design and Implementation (OSDI), November 1994.

....code reuse, and ease of programming. Notice also that al though the parallel I O libraries (e.g. 6] built on top of parallel file systems have the potential to provide both ease of use and high performance through the use of advanced I O optimization techniques such as col lective I O [14,20], and array chunking [26] their ex tensibility is severely limited by the design principles and the programming language used in their implemen tation [19] At the other end of the spectrum are database man agement systems (DBMS) They provide a layer on top of file systems, which is ....

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proc. the 1994.

....are run on a 16 node IBM SP2 at CPDC of Northwestern University. Each node of the SP2 is RS 6000 Model 390 processor. The HPSS is located at SDSC. We also use the performance model in the previous section to analyze the results that we obtained. 4. 1 Remote Collective I O Collective I O [9, 2, 5] is a technique to glean individual I O requests of each processor into a global picture and then decide how the combined I O requests should be best performed. The two phase I O [9, 2] is one of significant implementation techniques of collective I O. In the two phase scheme, the data is accessed ....

D. Kotz. Disk-directed i/o for mimd multiprocessors. In Proc. the 1994.

....some I O systems perform best when the number of simultaneous accesses is limited [KRY 93, NIT 92] These systems in particular benefit from this approach. 2.3. Disk directed I O Disk directed I O (DDIO) is a combination of a number of other techniques for data transfer in parallel I O systems [KOT 97] DDIO was developed after both the data sieving and two phase techniques, and it relies on both noncontiguous and collective I O primitives in the file system. Additionally the I O servers must be able to map file locations into disk block positions and must be capable of reasonably predicting ....

KOTZ D., "Disk-directed I/O for MIMD Multiprocessors", ACM Transactions on Computer Systems, vol. 15, num. 1, 1997, p. 41--74, ACM Press.

....component in the system, and few of them provide a programming environment, leading to simple static partitioning approaches in the programs. More advanced parallel file systems have specified higher level interfaces to data via collective I O (a similar concept is expressed with two phase I O) Kot94, CBH 94] In the original paper, Kotz found that many scientific codes show tremendous improvement by aggregating I O requests and then shipping them to the underlying I O system; the I O nodes can then schedule the requests, and often noticeably increase delivered bandwidth. Because requests ....

David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61--74. USENIX Association, November 1994. 104

....number of timesteps has been completed. Efficient transfer of this output from memory to local or remote disks is very important to achieve high performance for such applications. Often the output data are disjoint subsets of a logically shared data set, encouraging the use of collective I O [1, 3, 7, 9, 11]. In this approach, all the processors cooperate to transfer data between disk and memory. Information about the on disk and in memory layouts of the data set is used to plan efficient file operations, and reorganize the data across the memory of the processors if necessary. Most collective I O ....

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the Symposium on Operating Systems Design and Implementation, November 1994. Through the Center for Simulation of Advanced Rockets (CSAR) at UIUC.

....the user program and the HSS. The user program can send query requests to MDMS to obtain information about data structures that will be accessed. Then, the user can use this information in accessing the HSS in an optimal manner, taking advantage of powerful I O optimizations like collective I O [36, 8, 23], prefetching [20] prestaging [13] and so on. The user program can also send access pattern hints to the MDMS and let the MDMS to decide the best I O strategy considering the storage layout of the data in question. These access pattern hints span a wide spectrum that contains inter processors ....

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proc. the 1994.

....to create a single, big I O request and sent to the storage system. As a result, the effective I O bandwidth is 2 P2 P3 Collective Request Access P1 P2 P3 P4 Multi Collective P2 P3 P2 P3 Figure 1: Overview of the MCIO. significantly increased. This optimization has many variants [19, 13, 22]; although any CIO technique can be used for MCIO, the one used in this study is two phase I O as implemented in ROMIO, a portable implementation of MPI IO from Argonne National Laboratories [26] ROMIO has been incorporated into several MPI libraries, including the MPI implementations of several ....

....is, each data item is delivered to its final destination) Several variations of the collective I O technique have been proposed in previous research. In node grouping [20] nodes making I O requests are partitioned into groups. Then, they take turns in performing the I O. In disk directed I O [13], one compute node sends the col P2 P3 P4 Figure 4: An access pattern involving four processors and eight files. lective request to all I O nodes. I O nodes optimize the access, perform the request, and send the data directly back to all compute nodes. 4 Multi Collective I O In this ....

Kotz, D. Disk-directed I/O for MIMD multiprocessors. In Proc. the 1994.

....increasing number of I O nodes. And the buffer which each I O node required will have similar variation. This implies that there is an optimal number of I O nodes to a given I O workload. These are consistent with previous I O study by S. J. Baylor, C. E. Wu, and D. Kotz, Nils in [BBY94] BW95] [DK94]. The I O rate(throughput) vary with I O workloads and number of I O nodes. There is a maximum I O rate for a given I O subsystem configuration 5 Conclusion Our discussion shows that the hybrid modelings technique (involves both analytic modeling and simulation modeling) is an effective way for ....

David Kotz, "Disk-directed I/O for MIMD multiprocessors", Dartmouth College Technical Report PCS-TR94-226.

....and parallel disks. One of the most important optimizations in MPI IO [7] an emerging parallel I O standard, is collective I O, an optimization that allows each processor to do I O on behalf of other processors if doing so improves the overall performance [5] This optimization has many variants [16, 13, 18]; the one used in this study is two phase I O as implemented in MPI 2 standard [14] In this implementation, I O is performed in two phases: an I O phase and a communication phase. In the I O phase, processors perform I O in a way that is most beneficial from the storage layout point of view. In ....

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proc. the 1994.

....to create a single, big I O request and sent to the storage system. As a result, the effective I O bandwidth is P2 P3 Collective Request Access P1 P2 P3 P4 Multi Collective P2 P3 P2 P3 Figure 1: Overview of the MCIO. significantly increased. This optimization has many variants [19, 13, 22]; although any CIO technique can be used for MCIO, the one used in this study is two phase I O as implemented in ROMIO, a portable implementation of MPI IO from Argonne National Laboratories [26] ROMIO has been incorporated into several MPI libraries, including the MPI implementations of several ....

....is, each data item is delivered to its final destination) Several variations of the collective I O technique have been proposed in previous research. In node grouping [20] nodes making I O requests are partitioned into groups. Then, they take turns in performing the I O. In disk directed I O [13], one compute node sends the col P2 P3 P4 Figure 4: An access pattern involving four processors and eight files. lective request to all I O nodes. I O nodes optimize the access, perform the request, and send the data directly back to all compute nodes. 4 Multi Collective I O In this ....

Kotz, D. Disk-directed I/O for MIMD multiprocessors. In Proc. the 1994.

....for over 12 months now, there are still several issues remained to be addressed in future work: Distributed Intelligent Storage: As a cluster grows in scale, a centralized storage system becomes a bottleneck. Distributed file system, such as xFS [30] and Frangipani [28] and disk directed I O [16] were proposed to address this issue. Ideally, we would want to have a distributed storage system that utilizes the idle capacity and bandwidth of hard disks in all display nodes. This storage system should also be intelligent in that it can performan content specific transformations on the data. ....

D. Kotz. Disk-directed I/O for MIMD multiprocessors. Proceedings of First Symposium on Operating System Design and Implementation, pp. 61-74, 1994.

....provides a model of a widely portable file system, but the portability and optimization needed for parallel I O cannot be achieved with the POSIX interface. The significant optimizations required for efficiency (e.g. grouping [15] collective buffering [1, 2, 16, 19, 22] and disk directed I O [13]) can only be implemented if the parallel I O system provides a high level interface supporting partitioning of file data among processes and a collective interface supporting complete transfers of global data structures between process memories and files. In addition, further efficiencies can be ....

David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994.

....can have aliases and object references, the applications involve disks accesses and persistent storage, and the runtime system we need to interface to works very di erently. Several runtime support libraries and le systems have been developed to support ecient I O in a parallel environment [13, 14, 22, 33]. They also usually provide a collective I O interface, in which all processing nodes cooperate to make a single large I O request. Our work is di erent in two important ways. First, we are supporting a much higher level of programming by involving a compiler. Second, our target runtime system, ....

David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the

....increased. One of the most important optimizations in MPI IO [11] an emerging parallel I O standard, is collective I O, an optimization that allows each processor to do I O on behalf of other processors if doing so improves the overall performance [9] This optimization has many variants [28, 26, 38]; the one used in this study is two phase I O as implemented in ROMIO, a portable implementation of MPI IO from Argonne National Laboratories [41] ROMIO has been incorporated into several MPI libraries, including the MPI implementations of several vendors (e.g. HP, SGI, NEC) and MPICH and LAM, ....

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proc. the

....for efficient execution. We also developed a hypergraph based strategy, which takes into account the pre existing distribution of input and output datasets across processors. Several run time support libraries and file systems have been developed to support efficient I O in a parallel environment [25,41,48,60]. These systems mainly focus on supporting regular strided access to uniformly distributed datasets, such as images, maps, and dense multi dimensional arrays. Our work, however, has focused on efficiently supporting parallel aggregation operations over subsets of irregular spatially indexed ....

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the

....small message overhead for instance, combine a set of small messages into a single large message as suggested in [5, 8] 5] describes the disk directed I O (DDIO) strategy for MIMD multiprocessors and presents the simulated I O performance of a variety of array distributions. The original DDIO [4] does not include optimization for small messages generated from CYCLIC distributions so the performance degrades by a large degree when data granularity is very small (8 bytes) and logical blocks of the file were laid out in consecutive physical blocks on disk. 5] addresses the problem by using ....

....are combined. Performance is not as high as in Figure 8, because Figure 8 involves sending receiving data that are contiguous in memory. However, when CYCLIC(K) is used in memory, messages are generally noncontiguous in memory, incurring extra cost for strided memory access. 6. Related Work [4] describes the disk directed I O strategy for MIMD multiprocessors and presents the I O performance of a variety of array distributions that obtained from simulation. Both [5] and [8] describe optimizations for small messages. As discussed in Section 1, 8] suggests multiple block disk directed ....

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceeding of the Symposium on Operating Systems Design and Implementation, pages 61--74, Nov. 1994.

....the disk accesses) is lost. The system can make up for this to some degree by using appropriate prefetching and write behind with the bu er cache, as done in Vesta. However, optimizations based on explicit information about how accesses from the di erent nodes interact should also be considered [23, 36]. In Vesta, le data is only cached at the I O nodes, to obviate the issue of maintaining coherence of a distributed cache. A recent study of application I O behavior shows that this may be an overly restrictive solution. Speci cally, limited client side caching can be highly bene cial for access ....

D. Kotz, \Disk-directed I/O for MIMD multiprocessors". In 1st Symp. Operating Systems Design & Implementation, pp. 61-74, USENIX, Nov 1994.

....p any further. 5.4 Extracting Aggregation Function Extracting aggregation function is straight forward. We just extract the body of the loop being analyzed. 6 Related Work Several runtime support libraries and file systems have been developed to support efficient I O in a parallel environment [17, 18, 32, 49]. These systems mainly focus on supporting regular strided access to uniformly distributed datasets, such as images, maps, and dense multi dimensional arrays. They also usually provide a collective I O interface, in which all processing nodes cooperate to make a single large I O request. Our work ....

David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61--74. ACM Press, November 1994.

....code to execute on or near the data servers, when that helps to reduce network traffic. There are many good reasons to allow applicationspecific functionality to run on processors other than the clients. Application specific data distribution patterns, perhaps to support disk directed I O [21], need to execute on the data servers. If the distribution of data across clients or across disks is dependent on the value of the data, moving that function to the data server can halve network traffic [22] Processors near the data servers can filter data in an application specific way, passing ....

D. Kotz, Disk-directed I/O for MIMD multiprocessors, ACM Trans. Comput. Syst. 15 (1) (1997) 41--74.

....help balance the load on storage devices. Scheduling can improve device throughput by reordering accesses for lowered positioning times, balance the use of resources over multiple requesting applications, and reduce processing overheads by coalescing distinct accesses into a single larger access [38, 55]. For example, algorithms have been developed for eiticient use of device parallelism and cache resources, given suiticient access pattern information or hints [7, 36, 54] Useful extensions would be to support multiple active streams with different information sources (type, access pattern, ....

KOTZ, D. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994.

No context found.

D. Kotz. Disk-directed I/O for MIMD Multiprocessors. In Proc. of the First USENIX Symp. on Operating Systems Design and Implementation, 1994.

No context found.

D. Kotz. Disk-Directed I/O for MIMD Multiprocessors. In Hai Jin, Tomi Cortes, and R. Buyya, editors, High Performance Mass Storage and Parallel Input/Output: Technologies and Applications, pages 513--535. IEEE Computer Society and John Wiley & Sons, 2001.

No context found.

David Kotz. Disk-directed I/O for MIMD multiprocessors. In Hai Jin, Toni Cortes, and Rajkumar Buyya, editors, High Performance Mass Storage and Parallel I/O: Technologies and Applications, chapter 35, pages 513--535. IEEE Computer Society Press and John Wiley & Sons, 2001.

No context found.

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994.

No context found.

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994.

No context found.

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994.

No context found.

David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61-- 74. USENIX Association, November 1994. Updated as Dartmouth TR PCS-TR94-226 on November 8, 1994.

No context found.

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994.

No context found.

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proc. the 1994.

No context found.

D. Kotz. Disk-directed I/O for MIMD multiprocessors. In Proc. the 1994.

No context found.

David Kotz. Disk-directed I/O for MIMD multiprocessors, February 1997. 69

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC