T. M. Madhyastha and D. A. Reed, \Exploiting Global Input/Output Access Pattern Classication," in Proceedings of SC '97: High Performance Computing and Networking, (San Jose), IEEE Computer Society Press, Nov. 1997.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....[33] has shown that using qualitative classi cation data about the access patterns enables better optimizations. In PPFS II, this information can be obtained by user supplied hints or via automatic classi cation techniques. PPFS II utilizes a set of trained, arti cial, neural networks (ANNs) [47] to classify application input output patterns. These automatic classi ers are implemented as Autopilot sensors which distribute the information to connected decision procedures. To maintain rule base portability, the user hints and the classi cation information obtained through sensors are ....

T. M. Madhyastha and D. A. Reed, \Exploiting Global Input/Output Access Pattern Classication," in Proceedings of SC '97: High Performance Computing and Networking, (San Jose), IEEE Computer Society Press, Nov. 1997.

....can provide the file system the necessary request pattern information (e.g. request sizes and strides) to dynamically adjust the disk striping layout. Because it is not always possible to obtain hints from application programs, PPFS II utilizes a set of trained, artificial neural networks (ANNs) [87] to classify application input output patterns automatically. These automatic classifiers are implemented as Autopilot sensor attached functions that distribute the information to connected decision procedures. To maintain rule base portability, the user hints and the classification information ....

.... RequestSize = LARGE ) CachingEnable = DISABLED; if ( ReadWriteMix = WRITEONLY Sequentiality = SEQUENTIAL ) CachingEnable = ENABLED; CacheSize = SMALL; BlockSize = LARGE; ReplacementPolicy = MOSTRECENTLYUSED; Figure 5.16: PPFS II adaptive caching rule base. [87] to classify access patterns as uniform or variable size, sequential or strided, and read only, write only, or read update write. Using the output of this sensor, the decision server can change caching and striping policies based on access pattern attributes and request rates. The decision server ....

T. M. Madhyastha and D. A. Reed, "Exploiting Global Input/Output Access Pattern Classification, " in Proceedings of SC '97: High Performance Computing and Networking, (San Jose), IEEE Computer Society Press, Nov. 1997. 170

....Caching Prefetching Policies. Parallel applications exhibit such a wide variation in access patterns that any one caching prefetching policy is unlikely to perform well for all applications [27] The file system must therefore either detect and automatically adapt to changing access patterns [16, 17] or provide an interface for the user to specify the access pattern or caching prefetching policy [2, 22] 10. File Preallocation. It is easy and inexpensive for a file system to provide a function to preallocate disk space for a file. If such a function is not provided, the MPI IO function MPI ....

T. Madhyastha and D. Reed. Exploiting Global Input/Output Access Pattern Classification. In Proceedings of SC97: High Performance Networking and Computing. ACM Press, November 1997.

.... subfiles, and forks [35] PPFS is a parallel file system developed at the University of Illinois for clusters of workstations [23] The developers use it as a testbed for research on various aspects of file system design, such as caching prefetching policies and automatic adaptive policy selection [29, 30]. PVFS is a parallel file system for Linux clusters developed at Clemson University [64] PVFS stripes files across the local disks of machines in a Linux cluster and provides the look and feel of a single Unix file system. The regular Unix commands, such as rm, ls, and mv, can be used on PVFS ....

....access pattern, the desired striping parameters, or the desired caching prefetching policies, or the file system can be designed to automatically detect and adapt its policies to the access pattern of the application. Various research efforts have demonstrated the benefits of such optimization [6, 29, 30, 41]. As mentioned above, hints can also be used to vary the sizes of temporary buffers used internally by the implementation for various optimizations. Choosing the right buffer size can improve performance considerably, as demonstrated in Section 13.5.2 and in [66] The hints mechanism in MPI IO ....

Tara M. Madhyastha and Daniel A. Reed. Exploiting global input/output access pattern classification. In Proceedings of SC97: High Performance Networking and Computing. ACM Press, November 1997.

.... by allowing applications to give prefetching hints to the operating system [Cao94, Patterson95, Mowry96] or by automatically discovering file access patterns in order to better predict which blocks to prefetch [Griffioen94, Lei97, Kroeger96] Techniques studied have included neural networks [Madhyastha97a] and hidden Markov models [Madhyastha97b] Our work differs from this work in three ways. First, we address only common case workloads that have sequential access patterns. Second, our model is parameterized by the file system s behavior such as caching strategy and file layout, and takes into ....

Tara M. Madhyastha and Daniel A. Reed. Exploiting global input /output access pattern classification. Proceedings of Supercomputing '97 (San Jose, CA), November 1997.

....we are currently working on a mechanism to automatically find a close approximation of a parallel application logical data access pattern at execution time. To achieve this automatic pattern detection, we investigate di#erent approaches, in particular artificial neural networks and Markov chains [19]. The use of on line statistics on disk accesses can also help our system to prefetch data. Information collected by the automatic pattern detector is used by a prophet to make predictions on future disk accesses and these predictions are sent to the prefetching mechanism. Another problem related ....

Tara M. Madhyastha and Daniel A. Reed. Exploiting global input/output access pattern classification. In Proceedings of SC97: High Performance Networking and Computing. ACM Press, November 1997.

....for decision making, and actuators implement decisions distributed decision making is the final component of closed loop adaptive control. Although one can implement decision procedures using algorithmic or decision table techniques, our experience with parallel resource management policies [11, 6] suggests that more flexible mechanisms are needed to accommodate complex, poorly understood policy spaces. Simply put, constructing decision tables presumes a deep understanding of the resource optimization space and the relation of system controls to locations in that space. In contrast to ....

T. M. Madhyastha and D. A. Reed. Exploiting Global Input/Output Access Pattern Classification. In Proceedings of Supercomputing '97, Nov. 1997.

....specialized) interfaces on top of PPFS. The advantage is that such interfaces would not have to cope with low level parallel I O details. Still regarding automatic tuning, there are promising results on runtime identification of patterns in the I O requests generated by parallel applications [Madhyastha 97] Since the runtime system might be able to automatically identify some collective I O behavior, this approach can be used to simplify the I O interface for parallel applications by omitting details that can be inferred by the system, providing therefore a better balance between performance and ....

Tara M. Madhyastha, and Daniel A. Reed. Exploiting Global Input/Output Access Pattern Classification. In Proceedings of Supercomputing '97, San Jose, CA, November 1997. http://www-pablo.cs.uiuc.edu/People/tara/sc97.ps.Z

....Caching Prefetching Policies. Parallel applications exhibit such a wide variation in access patterns that any one caching prefetching policy is unlikely to perform well for all applications [26] The file system must therefore either detect and automatically adapt to changing access patterns [15, 16] or provide an interface for the user to specify the access pattern or caching prefetching policy [2, 21] 10. File Preallocation. It is easy and inexpensive for a file system to provide a function to preallocate disk space for a file. If such a function is not provided, the MPI IO function MPI ....

T. Madhyastha and D. Reed. Exploiting Global Input/Output Access Pattern Classification. In Proceedings of SC97: High Performance Networking and Computing. ACM Press, November 1997.

....process can verify hint accuracy and educate users about application behavior by reporting actual qualitative behavior. To support automatic, qualitative classification of resource use, we have developed a suite of trained artificial neural networks (ANNs) and hidden Markov models (HMMs) [13,11]. Within the Autopilot framework, our ANN HMM classification software operates as a group of sensor attached functions, intercepting application input output parameters, and reporting qualitative classifications to sensor clients. Based on our experience classifying input output patterns [13,11] ....

....[13,11] Within the Autopilot framework, our ANN HMM classification software operates as a group of sensor attached functions, intercepting application input output parameters, and reporting qualitative classifications to sensor clients. Based on our experience classifying input output patterns [13,11], we believe that both ANNs and HMMs are necessary each is suited to a different type of classification problem. After offline training, ANNs can efficiently classify access streams in real time (e.g. identifying input output requests as strided and read only) However, they cannot identify ....

Tara M. Madhyastha and D. A. Reed. Exploiting Global Input/Output Access Pattern Classification. In Proceedings of Supercomputing '97, November 1997.

....As our extensive analysis of input output dynamics [6, 35, 32] has shown, the parallel input output patterns in emerging applications are both irregular and dynamic. Because the interactions between these applications and the file system software change during and across application executions [19], it is difficult or impossible to determine a globally optimal input output configuration or to statically configure runtime systems and resource management policies for parallel input output. Hence, parallel input output optimization provides an excellent test of an adaptive, closed loop ....

....II is designed to work atop either parallel systems or PC and workstation clusters, providing a flexible testbed for highperformance input output experiments. To explore automatic, qualitative classification of resource use, we have developed a suite of trained artificial neural networks (ANNs) [19] and hidden Markov models (HMMs) 20] that are implemented as Autopilot sensors. ANNs can efficiently classify access streams in real time. In contrast, HMMs build a probabilistic model of the access pattern using prior execution training. This generality allows HMMs to classify arbitrary access ....

Madhyastha, T. M., and Reed, D. A. Exploiting Global Input/Output Access Pattern Classification. In Proceedings of SC '97: High Performance Computing and Networking (San Jose, Nov. 1997), IEEE Computer Society Press.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC