C. S. Ellis and D. Kotz. Prefetching in file systems for MIMD multiprocessors. In Proc. the 1989.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....have presented challenging problems to computational scientists. A major problem is the I O performance when many datasets are stored and frequently accessed. Over the years, scientists have developed techniques for I O optimizations such as collective I O [14, 38, 23, 10] prefetching [15, 13], data sieving [38, 10] caching [8] and so on. But the performance problem still persists because data intensive applications are Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL 60208, choudhar ece.nwu.edu generating more data and the usage of these ....

C. S. Ellis and D. Kotz. Prefetching in file systems for mimd multiprocessors. In Proc. the 1989.

....of current solutions in the market. For example, parallel file systems [11, 30, 9] might be effective for applications whose I O access patterns fit a few specific forms. They achieve impressive performance for these applications by utilizing smart I O optimization techniques such as prefetching [19], caching [24, 6] and parallel I O [17, 12] However, there are serious obstacles preventing the parallel file systems from becoming a global solution to the data management problem. First of all, user interfaces of the file systems are in general low level [22] allowing the users to express ....

....data. The user program can send query requests to MDMS to obtain information about data structures that will be accessed. Then, the user can use this information in accessing the HSS in an optimal manner, taking advantage of powerful I O optimizations like collective I O [35, 8, 23] prefetching [19], prestaging [14] and so on. The user program can also send access pattern hints to the MDMS and let the MDMS to decide the best I O strategy considering the storage layout of the data in question. These access pattern hints span a wide spectrum that contains inter processors I O access patterns, ....

[Article contains additional citation context not shown here]

C. S. Ellis and D. Kotz. Prefetching in file systems for mimd multiprocessors. In Proc. the 1989.

....3. There are opportunities for improved I O using program transformations. Abu Sufah [1] showed that improving locality can benefit program performance when using virtual memory. Prefetching also has been shown to provide performance improvements in some cases, when done by the operating system [13] or suggested by the application level [16] Currently, prefetching and staging transformations are often done by hand to improve performance on large data sets. 4. Useful information is available to the compiler. The determination of interprocessor communication for both regular and irregular ....

....for both regular and irregular problems by the Fortran D compiler supports the feasibility of identifying I O patterns at compile time. Identifying access patterns at run time on parallel machines can be very costly and may introduce unnecessary demands on and conflicts for system resources [13]. The thesis of this research is that a compiler can transform an appropriate in core algorithm to operate on out of core data through I O insertion and data set .splitting. Data set splitting refers to the staging of data through the steps of the computation. We illustrate the programming ....

[Article contains additional citation context not shown here]

David Kotz and Carla Schlatter Ellis. Prefetching in file systems for MIMD multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 1(2):218-230, April 1990.

....of current solutions in the market. For example, parallel file systems [9, 31, 11] might be effective for applications whose I O access patterns fit a few specific forms. They achieve impressive performance for these applications by utilizing smart I O optimization techniques such as prefetching [20], caching [25, 7] and parallel I O [18, 12] However, there are serious obstacles preventing the parallel file systems from becoming a global solution to the data management problem. First of all, user interfaces of the file systems are in general low level [24] allowing the users to express ....

....HSS. The user program can send query requests to MDMS to obtain information about data structures that will be accessed. Then, the user can use this information in accessing the HSS in an optimal manner, taking advantage of powerful I O optimizations like collective I O [36, 8, 23] prefetching [20], prestaging [13] and so on. The user program can also send access pattern hints to the MDMS and let the MDMS to decide the best I O strategy considering the storage layout of the data in question. These access pattern hints span a wide spectrum that contains inter processors I O access patterns, ....

[Article contains additional citation context not shown here]

C. S. Ellis and D. Kotz. Prefetching in file systems for MIMD multiprocessors. In Proc. the 1989.

....in a natural manner and still achieve good performance even on out of core problem sizes. 5 Related Work Much related work has depended on the use of an explicit I O interface by the programmer. On the OS side, this work includes the automatic detection of file access patterns in the file system [1, 10, 11, 12, 14, 15, 17], as well as the use of access patterns supplied directly by the application using an I O type of interface [22, 26, 30, 4] For compilers it involves analysis to move explicit I O calls back and change them to asynchronous I O calls instead [25] While some of the OS policies developed may be ....

David Kotz and Carla Schlatter Ellis. Prefetching in file systems for MIMD multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 1(2):218 230, April 1990.

....which cannot be satisfied by secondary storage devices and for applications which cannot afford the cost or system complexity of a large number of disk drives. There has been a considerable amount of work in addressing the flow of data to and from secondary storage devices (e.g. magnetic disks) [1, 2, 3, 4, 5, 6, 7, 8, 9]. There has also been a significant amount of work on the management of large scale data in a storage hierarchy involving tertiary storage devices (e.g. tapes devices) 10, 11, 12, 13, 14] Striping has been studied to improve the response time of tertiary storage devices [15, 16] The ....

C. S. Ellis and D. Kotz. Prefetching in file systems for MIMD multiprocessors. In Proc. of the 1989.

.... incorporating Markov sources of examples was developed in [1] Recent empirical work on prefetching include a pattern matching approach to prediction [30] computing various firstorder statistics for prediction [32] a growing order Markov predictor [24] prefetching in a parallel environment [22], and research projects at a lower level of abstraction including compiler directed prefetching [6,29,31] In this paper, we develop a randomized algorithm for pure prefetching and show its optimality in the limit under the following analysis strategy. Putting no restrictions on the generator of ....

D. F. Kotz and C. S. Ellis, "Prefetching in File Systems for MIMD Multiprocessors," IEEE Transactions on Parallel and Distributed Systems 1 (April 1990), 218--230.

....when arbitrary range queries are allowed. Jagadish et al. [23] investigated the problem of efficient organization of a data warehouse on secondary storage. There has been a considerable amount of work in addressing the flow of data to and from secondary storage devices (e.g. magnetic disks) [6, 9, 11, 34, 16, 29, 31, 32, 5]. There has also been a significant amount of work on the management of large scale data in a storage hierarchy involving tertiary storage devices (e.g. tapes 8 9 devices) 37, 22, 24, 25, 30] Striping has been studied to improve the response time of tertiary storage devices [15, 19] Also, ....

C. S. Ellis and D. Kotz. Prefetching in file systems for MIMD multiprocessors. In Proc. of the

....in a natural manner and still achieve good performance even on out of core problem sizes. 5 Related Work Much related work has depended on the use of an explicit I O interface by the programmer. On the OS side, this work includes the automatic detection of file access patterns in the file system [1, 10, 11, 12, 14, 15, 17], as well as the use of access patterns supplied directly by the application using an I O type of interface [22, 26, 30, 4] For compilers it involves analysis to move explicit I O calls back and change them to asynchronous I O calls instead [25] While some of the OS policies developed may be ....

David Kotz and Carla Schlatter Ellis. Prefetching in file systems for MIMD multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 1(2):218--230, April 1990.

....which cannot be satisfied by secondary storage devices and for applications which cannot afford the cost or system complexity of a large number of disk drives. There has been a considerable amount of work in addressing the flow of data to and from secondary storage devices (e.g. magnetic disks) [3, 4, 5, 10, 11, 14, 15, 16, 2]. There has also been a significant amount of work on the management of large scale data in a storage hierarchy involving tertiary storage devices (e.g. tapes devices) 19, 12, 17, 18, 7] Striping has been studied to improve the response time of tertiary storage devices [13, 6] The Department ....

C. S. Ellis and D. Kotz. Prefetching in file systems for MIMD multiprocessors. In Proc. of the

....with memory distributed among processor nodes. Some results show that prefetching often reduces the total execution time. As a matter of fact, the hit ratio is only a rough indicator of overall performance of a caching system since it tends to be optimistic and ignores prefetching overhead [24]. read ahead Communication between clients and servers (or in distributed systems in general) is one of the main overheads in a file system. Hence, I O requests are packaged, and level locks and resources are managed in groups. Read ahead reads new blocks in advance when a minimum threshold is ....

David F. Kotz and Carla Schlatter Ellis. Prefetching in file systems for MIMD multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 1(2):218--230, April 1990.

....HSS. The user program can send query requests to MDMS to obtain information about data structures that will be accessed. Then, the user can use this information in accessing the HSS in an optimal manner, taking advantage of powerful I O optimizations like collective I O [18, 5, 11] prefetching [10], prestaging [7] and so on. The user program can also send access pattern hints to the MDMS and let the MDMS to decide the best I O strategy considering the storage layout of the data in question. These access pattern hints span a wide spectrum that contains interprocessors I O access patterns, ....

C. S. Ellis and D. Kotz. Prefetching in file systems for MIMD multiprocessors. In Proc. the 1989 International Conference on Parallel Processing, pages I:306--314, St. Charles, IL, August 1989. Pennsylvania State Univ. Press.

....problem use either high performance parallel file systems or database management systems. Parallel file systems have been built to exploit the parallel I O capabilities provided by modern architectures and achieve this goal by adopting smart I O optimization techniques such as prefetching [1], caching [2] and parallel I O [3] However, there are serious obstacles preventing the file systems from becoming a real solution to the high level data management problem. First of all, user interfaces of the file systems are low level which forces the users to express details of access ....

C. Ellis and D. Kotz. Prefetching in File Systems for MIMD Multiprocessors. In International Conference on Parallel Processing, volume 1, pages 306--314, August 1989.

.... incorporating Markov sources of examples was developed in [1] Recent empirical work on prefetching includes a pattern matching approach to prediction [30] computing various first order statistics for prediction [32] a growing order Markov predictor [24] prefetching in a parallel environment [22], and research projects at a lower level of abstraction including compiler directed prefetching [6, 29, 31] In this paper, we develop a randomized algorithm for pure prefetching and show its optimality in the limit under the following analysis strategy. Putting no restrictions on the generator of ....

D. F. Kotz and C. S. Ellis, Prefetching in File Systems for MIMD Multiprocessors, IEEE Transactions on Parallel and Distributed Systems, Vol. 1, April 1990, pp. 218--230.

....have proposed schemes for exploiting access pattern information both in sequential and parallel systems [25, 15] In [15] Kotz uses pattern predictors to anticipate an application s future access patterns. A variety of data management strategies for parallel input output systems are explored in [16, 8, 17]. 1.3 Organization The rest of this thesis presents the design of our PPFS and its use in input output experiments. Chapter 2 presents the logical organization of PPFS. Chapter 3 describes the current implementation of PPFS. In Chapter 4, we discuss our experiences with using PPFS for two ....

Ellis, C. S., and Kotz, D. Prefetching in File Systems for MIMD Multiprocessors. In Proceedings of the 1989 International Conference on Parallel Processing (August 1989), pp. I:306--314. 58

....to prefetching as described in Section 2.2 is also based on monitoring past user behavior; we develop a systematic approach to prefetching based on data compression principles. Prefetching has also been studied in other systems environments. Prefetching in a parallel environment is studied in [KoE]. More recently, the possibility of predictive prefetching in mobile computing has been studied by Kuenning et al. KPR] Cao et al. study integrated prefetching and caching strategies [CFK] in file systems assuming full knowledge of future requests; the concern in this case is to decide when to ....

D. F. Kotz and C. S. Ellis, "Prefetching in File Systems for MIMD Multiprocessors, " IEEE Transactions on Parallel and Distributed Systems 1 (April 1990), 218--230.

....unlikely to increase dramatically. As a consequence, there have been a number of recent proposals for the use of multiple disks to form high performance I O subsystems [9, 16, 18] Performance evaluation of different multiple disk systems, and associated management strategies have been studied in [19, 12, 17, 5, 6, 10], for example. A number of analytic studies of I O performance for specific computational problems have been undertaken previously in [11, 20, 8, 1, 21, 2, 23, 22, 14, 4] This paper suggests an analytic framework for the study of I O parallelism by undertaking a specific case study of ....

....drops as the number of disks increases past a limit. As the cache size increases, the parallelism approaches the number of disks. The deterministic model shows a similar trend as well. 7 Conclusions Choosing a prefetching strategy that maximizes I O performance is, in general, a difficult task [10]. In this study, we investigated the performance of two prefetching strategies that can be used with external mergesort. To compare the cache requirements of the two strategies, a Markov model was developed for each strategy. Closed form expressions were derived for the average I O 24 parallelism ....

D. F. Kotz and C. S. Ellis. Prefetching in File Systems for MIMD Multiprocessors. IEEE Transactions on Parallel and Distributed Computing, 1(2):218--230, 1990.

....This is called prefetching of data blocks. The most common technique used for caching is the one block lookahead or OBL. This strategy prefetches the (i 1)th block when the i th block is prefetched [Smi78] However in the parallel file systems, the prefetching strategy depends upon many factors [Dav90, EK89, KE92, KE91, Kot91] These include what to fetch, when to fetch, which disk to fetch from, which processor should make the decision of prefetching and where the data should be stored. The common prefetching strategy used for the uniprocessors may not work here because of different file ....

....accessing the data from several blocks, it may be beneficial to prefetch the blocks that can be accessed in near future. Thus the many block lookahead (MBA) might be a good replacement for (OBA) Kotz et al. have performed experiments on the parallel file system testbed, the RAPID Transit System [Dav90] According to their results, the prefetching contributes to the decrease in the execution time. The improvement ranged from 15 percent to 70 percent. Similar work was carried out by French et al. FPD91a, FPD91b] They studied the effect of prefetching and caching on the performance of an Intel ....

David Kotz and Carla Ellis. Prefetching in File Systems for MIMD Multiprocessors. IEEE Transactions on Parallel and Distributed Systems, pages 218--230, April 1990.

....of the spectrum there are highperformance parallel file systems (e.g. Intel s PFS [25] and IBM s Vesta [8] that have been built to exploit the parallel I O capabilities provided by modern architectures. They achieve this goal by adopting smart I O optimization techniques such as prefetching [17], caching [22,5] and parallel I O [15,10] However, there are serious obstacles preventing the file systems from becoming a real solution to the high level data management problem. First of all, user interfaces of the file systems are low level [21] They force the users to express access ....

....based on access patterns (hints) provided by the application. By comparing the storage pattern and access pattern 4 A. Choudhary et al. Data Management for Large Scale Scientific Computations of a data set, the MDMS can, for example, advise the HSS to perform collective I O [14] or prefetching [17] for the data set in question. ffl It stores information about the pending access patterns. It utilizes this information in taking some global decisions, possibly involving data sets from multiple applications (e.g. staging a number of related files from tape sub system to disk sub system, or ....

[Article contains additional citation context not shown here]

C. S. Ellis and D. Kotz. Prefetching in file systems for MIMD multiprocessors. In Proc. the 1989 International Conference on Parallel Processing, pages I:306--314, St. Charles, IL, August 1989. Pennsylvania State Univ. Press.

....of current solutions in the market. For example, parallel file systems [10, 29, 8] might be effective for applications whose I O access patterns fit a few specific forms. They achieve impressive performance for these applications by utilizing smart I O optimization techniques such as prefetching [18], caching [23, 6] and parallel I O [16, 11] However, there are serious obstacles preventing the parallel file systems from becoming a global solution to the data Visualization Analysis Simulation Archive Data Decomposition Mesh Adjust Generation Parameters Domain Cycle Figure 1: A ....

....HSS. The user program can send query requests to MDMS to obtain information about data structures that will be accessed. Then, the user can use this information in accessing the HSS in an optimal manner, taking advantage of powerful I O optimizations like collective I O [34, 7, 22] prefetching [18], prestaging [13] and so on. The user program can also send access pattern hints to the MDMS and let the MDMS to decide the best I O strategy considering the storage layout of the data in question. These access pattern hints span a wide spectrum that contains inter processors I O access patterns, ....

[Article contains additional citation context not shown here]

C. S. Ellis and D. Kotz. Prefetching in file systems for mimd multiprocessors. In Proc. the 1989 International Conference on Parallel Processing, pages 306--314, 1989.

....stored on different disk drives. Kim [8] has proposed disk synchronization, in which a file is byte wise distributed across the disks, and the disks synchronously read a block. Reddy and Banerjee [13, 14, 15] compare the performance of disk synchronization and disk striping. The Bridge filesystem [3, 2, 9] is a parallel striped file system implemented on a BBN butterfly. Files in Bridge are striped across multiple disks in a regular pattern. The Bridge file system can support highly parallel block file reads and file appends. Based on these operations, the authors can implement a highly parallel ....

....on a message passing parallel computer. 2 Parallel Striped Files In this section, we review the structure of a parallel striped file, discuss the operations on the file, and describe the difficulty of performing insertions or deletions in the middle of the file. We will use the Bridge filesystem [3, 2, 9] as a concrete example. The Bridge file system consists of a number of Local File Systems (or LFSs) distributed throughput the parallel computer. Each LFS is managed by a single processor. The blocks of a file are distributed in round robin fashion among L LFSs, d 0 ; d 1 ; dL Gamma1 . If ....

[Article contains additional citation context not shown here]

D. Kotz and C.S. Ellis. Prefetching in file systems for MIMD multiprocessors. IEEE Trans. on Parallel and Distributed Systems, 1(2):218--230, 1990.

....sequential access, and the dB tree index provides fast and highly concurrent random access. We note that in this paper, we only address the issues of indexing, accessing, and modifying an indexed striped file. We don t address issues of distributing the data once it is retrieved [1, 3] caching [9, 10, 4], and other file system issues. 3 The dE tree Our approach to maintaining a striped file is to break it into extents, then keep an index into the extents. Therefore, one major problem that we need to solve is that of maintaining a distributed index. In this section, we discuss our highly ....

C.S. Ellis and D. Kotz. Prefetching in file systems for mimd multiprocessors. In Int'l Conf. on Parallel Processing, pages I:306--314, 1989.

....be to customize storage access scheduling and cachereplacement decisions. Prefetching and write behind can hide access latency by overlapping accesses with computation or other accesses, increasing exploitation of parallel storage resources, and mi grating storage accesses to less busy periods [39]. Caching policies can lessen the cost of accessing storage and can help balance the load on storage devices. Scheduling can improve device throughput by reordering accesses for lowered positioning times, balance the use of resources over multiple requesting applications, and reduce processing ....

KOTZ, D., AND ELLIS, C. S. Prefetching in file systems for MIMD multiprocessors. IEEE Transactions on Parallel and Distributed Systems 1, 2 (April 1990), 218-230. 13

....and the average time to perform an I O operation) We explore why it is not trivial to translate these gains into much better overall performance. This technical report, although still available on hardcopy from Duke University, has been superceded by [EK89] and later published more thoroughly as [KE90]. For yet another presentation and many more results, see also [Kot91, KE91b, KE92b, KE91a, KE92a, Kot92] For more information, write to Technical Reports Department of Computer Science Duke University Durham, North Carolina 27708 0129 techreports cs.duke.edu ....

David Kotz and Carla Schlatter Ellis. Prefetching in file systems for MIMD multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 1(2):218--230, April 1990.

No context found.

C. S. Ellis and D. Kotz. Prefetching in file systems for MIMD multiprocessors. In Proc. the 1989.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC