Apratim Purakayastha, Carla Schlatter Ellis, David Kotz, Nils Nieuwejaar, and Michael Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, pages 165--172, April 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....in Figure 1(a) illustrate several language features. The shape S is logically laid out as a 256 Theta 128 array of VPs. Stored at each VP is a single integer x and an array y containing 10 doubles. The variables i, sum, and scalarFile are scalars. shape [256] 128]S; int:S x; double:S y[10]; int i; double sum; FILE scalarFile; sum = 0; Scalar code for (i = 0; i 10; i ) Also scalar with (S) Perform in parallel x = y[i] where (x 100) Only some VPs will do this x = x 100; a) b) Figure 1: a) Sample C declarations. b) Scalar and ....

....general parallel file operations. We say files relying on this generality use Independent Buffering Mode (IB) because each VP s stream is managed separately. Many debugged, production quality, data parallel applications do not require such generality; they perform very 5 regular file operations [1, 2, 5, 7, 8, 10]. When regular operations are performed, all VPs read or write the same amount of data at the same time. In such instances, a single value can act as the file pointer for all the VP streams. During a write, each VP copies data from its parallel variable into its file buffer. Under the assumption ....

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, April 1995.

....instead, they instrumented the Concurrent File System (CFS) library routines called by all applications, and studied the file related activity of all applications together. A similar study on the CM 5 at the National Center for Supercomputing Applications was performed by Purakayastha et al. [27]. The results of these two studies are compared and contrasted in [23] They found that file sizes were large, request sizes were fairly small, data was accessed in sequence but with strides, and I O was dominated by writes. Crandall et al. 8] instrumented and analyzed the I O characteristics of ....

A. Purakayastha, C. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing Parallel FileAccess Patterns on a Large-Scale Multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, pages 165--172, April 1995.

.... parallel file systems both from the academia and industry have emerged [10] Apart from presenting the user convenient interface for indicating parallel I O accesses from within her program, those file systems also support several I O modes which take care of the commonly seen file access patterns [28] in explicitly or implicitly parallelized scientific codes. These patterns can be characterized as regular and non consecutive I O accesses [18] and can be described by strided interfaces and their variants. Recently a number of run time libraries for out of core computations and a few file ....

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, pages 165-172, April 1995.

....(BLOCK, and ( BLOCK) from left to right. The server meshes are indicated in the figure. 4 Related Work To design a good parallel i o subsystem we must understand its intended usage. Thus there are many recent studies characterizing dynamic i o patterns in scientific applications ([Kotz94a, Thakur95, Pura94, Pasquale94, Miller91]) They show that many scientific applications have regular patterns of i o behavior, such as physical periodicity in strided access to multidimensional arrays, or temporal periodicity in checkpoint and restart operations. In both cases all compute nodes participate in collective i o operations to ....

A. Purakayastha, C. Ellis, D. Kotz, N. Nieuwejaar, and M. Best, Characterizing parallel file-access patterns on a large-scale multiprocessor, Duke TR CS-1994-33, 1994.

....a proper discussion of all the work related to Panda; we will limit our discussion to papers from the HPCC community that appeared in 1993 or later and that focus on parallel implementations of collective i o. Recent studies characterizing dynamic i o patterns in scientific applications [Kotz94a, Purakayastha94, Pasquale94] show that many scientific applications have regular patterns of i o behavior, such as physical periodicity in strided access to multidimensional arrays, or temporal periodicity in checkpoint and restart operations. In both cases all application compute nodes participate in i o ....

A. Purakayastha, C. Ellis, D. Kotz, N. Nieuwejaar, and M. Best, Characterizing Parallel File-Access Patterns on a Large-Scale Multiprocessor, Duke University Technical Report CS-199433, October 1994.

....5 which has fine grained I O spread throughout the application. The second application uses large grained I O operations. Finally, the conclusions are presented. 2 Related Work There have been several studies of parallel I O for applications using real data. Some examples of these are found in [10, 25, 29, 30, 34, 35]. These studies look at traces of actual parallel applications doing I O using specific parallel I O libraries and architectures. Characterizing well understood parallel programs under controlled conditions facilitates development of optimization techniques. Parallel I O systems such as PIOUS[27] ....

A. Purakayastha, C. Schlatter Ellis, D. Kotz, N. Nieuwejaar, and M. Best, "Characterizing Parallel File-Access Patterns on a Large-Scale Multiprocessor," In proceedings of Ninth International Parallel Processing Symposium, pp. 165-172, Santa Barbara, CA, 1995.

....The volume of metadata describing the mode switches and VP activity in every IB segment would be huge. As we see in Section 5.2, IB s cost relative to NB and CB is not prohibitive. Finally, files used by most data parallel applications would remain in NB or CB modes during their lifetimes [5, 7, 14, 16, 21], and Stream s optimistic approach matches their needs without becoming overly complex. 1 Note that operations allowed in NB mode form a subset of those allowed in CB. NB CB IB Figure 4: The segmented nature of a Stream file. The first segment is written using NB mode, and VP blocks of size ....

.... built using a simple utility program in some situations (e.g. if a file has a different distribution than the program that will read it) Because most data parallel applications rely on regular array oriented I O, the use of NB as an interface to the outside world should work in most situations [5, 7, 14, 16, 21]. Files with CB and IB segments require explicit conversions. A file containing a CB or IB segment can be converted to NB using a high level C program. The program has each VP reading its existing stream and writing its data, or a fixed value upon reaching EOF on its input, in NB mode. 7 ....

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, April 1995.

....distribution of request delays and that current parallel file systems respond to parallel application input output patterns in non scalable ways. 1 Introduction Input output for scalable parallel systems continues to be the major performance bottleneck for many large scale scientific applications [2, 14]. Market forces are increasing the disparity between processor and disk system performance, exacerbating the already difficult problem of This work was supported in part by the Defense Advanced Research Projects Agency under DARPA contracts DABT63 94 C0049 (SIO Initiative) DAVT63 91C 0029 and ....

....under NASA Contract NAG 1 613. achieving high performance for applications with large input output components. Moreover, most current parallel file systems were constructed as extensions of workstation file systems and optimized for large, sequential data transfers. Recent experimental studies [14, 2, 22, 23, 16] have shown that parallel applications have much more complex access patterns, with greater spatial and temporal variability, than first suspected. Although there is a large, complementary body of experimental data on disk behavior [19, 18] for sequential file systems, there is much less ....

[Article contains additional citation context not shown here]

Purakayastha, A., Ellis, C. S., Kotz, D., Nieuwejaar, N., and Best, M. Characterizing Parallel File Access Patterns on a Large-Scale Multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium (Apr. 1995), pp. 165--172.

....in every IB segment potentially would be huge. IB s performance relative to the other modes is good enough that the extra overhead for allowing more file segments is not justified. Finally, files used by most data parallel applications would remain in NB or CB modes during their lifetimes [5, 7, 15, 17, 19, 23], and Stream s prioritization of the highest performance modes matches their needs without becoming overly complex. Associated with each Stream data file is a second metafile, whose name is the data file name with the suffix .meta. This file contains information about the file such as bNB ; ....

.... built using a simple utility program in some situations (e.g. if a file has a different distribution than the program that will read it) Because most data parallel applications rely on regular array oriented I O, the use of NB as an interface to the outside world should work in most situations [5, 7, 15, 17, 19, 23]. Files with CB and IB segments require explicit conversions. A file containing a CB or IB segment can be converted to one containing a single NB segment using a high level C program. The program has each VP reading its existing stream and writing its data, or a fixed value upon reaching EOF on ....

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, pages 165--172, April 1995.

....concluded that I O behavior is regular, recurrent, and predictable [8, 10] characteristics that were attributed to the iterative nature of such applications. In contrast to the results on vector systems, recent analyses on the Intel Paragon XP S [4] the Intel iPSC 860 [6] and the CM 5 [12], showed greater irregularity in I O access patterns, with the majority of file requests being small but with the greatest data volume generated by a few large requests. To accommodate the variability of parallel I O, a growing number of parallel system vendors provide extended I O APIs that ....

Purakayastha, A., Ellis, C. S., Kotz, D., Nieuwejaar, N., and Best, M. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium (April 1995), pp. 165--172.

....the next section, we present background relating to software support for outof core computation. 1. 2 Background The awareness of the importance of I O subsystems on the overall performance of parallel machines has spurred a large research interest in various aspects of out ofcore applications [24, 16, 78, 29, 69]. In this section, we review these research efforts in the areas of models, program synthesis, file systems, languages, compilers, etc. 4 1.2.1 Models and Algorithm Design A model of parallel computation is an abstraction of a parallel machine. An ideal parallel model should be simple enough to ....

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, pages 165-- 172, April 1995.

....is unique in recording individual read and write requests in live, multiprogramming, parallel workloads. We have so far completed characterization studies on an Intel iPSC 860 at NASA s Ames Research Center [1] and on a Thinking Machines CM 5 at the National Center for Supercomputing Applications [2]. On both systems we addressed a similar set of questions: ffl What did the job mix look like: How many jobs were run concurrently How many processors did each job use ffl How many files were read and written What were their sizes 1 More about CHARISMA may be found at ....

.... using Pablo to analyze and characterize specific applications, and Crandall et al. performed such an analysis on three scientific applications [20] As part of the CHARISMA project, we have traced parallel I O requests by a live, production mix of user programs on an Intel iPSC [1] and on a CM 5 [2]. No other study has included more than one machine or programming platform. 2.2 Existing Parallel File Systems A single, coherent model of parallel file access has not yet emerged. Parallel I O models are often closely tied to a particular machine architecture as well as to a programming model. ....

Apratim Purakayastha, Carla Schlatter Ellis, David Kotz, Nils Nieuwejaar, and Michael Best, "Characterizing parallel file-access patterns on a large-scale multiprocessor", in Proceedings of the Ninth International Parallel Processing Symposium, Apr. 1995, pp. 165--172.

....tracing project. We defined a generic set of trace records that logged events such as open, close, read, write, truncate extend, link unlink, etc. The actual format of the records differed slightly dependingon the platform and programming model. Detailed formats of event records are listed in [27]. We specifically considered user program I O only to and from the SDA. Serial NFS I O was not traced because we expected that it would have much less data traffic due to limited bandwidth. We instrumented the run time CMF I O libraries to collect traces. The normal I O library was replaced by ....

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. Technical Report CS-1994-33, Dept. of Computer Science, Duke University, October 1994.

No context found.

Apratim Purakayastha, Carla Schlatter Ellis, David Kotz, Nils Nieuwejaar, and Michael Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, pages 165--172, April 1995.

No context found.

Apratim Purakayastha, Carla Schlatter Ellis, David Kotz, Nils Nieuwejaar, and Michael Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. Technical Report CS-1994-33, Dept. of Computer Science, Duke University, October 1994.

....tracing project. We defined a generic set of trace records that logged events such as open, close, read, write, truncate extend, link unlink, etc. The actual format of the records differed slightly dependingon the platform and programming model. Detailed formats of event records are listed in [27]. We specifically considered user program I O only to and from the SDA. Serial NFS I O was not considered because we expected that it would have much less data traffic due to limited bandwidth. We instrumented the run time CMF I O libraries to collect traces. The normal I O library was replaced ....

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. Technical Report CS-1994-33, Dept. of Computer Science, Duke University, October 1994.

No context found.

Apratim Purakayastha, Carla Schlatter Ellis, David Kotz, Nils Nieuwejaar, and Michael Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, April 1995. To appear.

No context found.

Apratim Purakayastha, Carla Schlatter Ellis, David Kotz, Nils Nieuwejaar, and Michael Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, pages 165--172, April 1995.

No context found.

Apratim Purakayastha, Carla Schlatter Ellis, David Kotz, Nils Nieuwejaar, and Michael Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium, pages 165--172, April 1995.

No context found.

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the 9th International Parallel Processing Symposium (IPPS '95), pages 165--172. IEEE Computer Society Press, 1995.

No context found.

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the 9th International Parallel Processing Symposium (IPPS '95), pages 165--172. IEEE Computer Society Press, 1995.

No context found.

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the 9th International Parallel Processing Symposium (IPPS '95), pages 165--172. IEEE Computer Society Press, 1995.

No context found.

A. Purakayastha, C. S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the 9th International Parallel Processing Symposium (IPPS '95), pages 165--172. IEEE Computer Society Press, 1995.

No context found.

A. Purakayastha, C.S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best, Characterizing Parallel File-Access Patterns on a Large-Scale Multiprocessor, Proc. Ninth Int'l Parallel Processing Symp., pp. 165 172, Apr. 1995.

No context found.

Purakayastha, A., Ellis, C., Kotz, D., Nieuwejaar, N., Best, M. Characterizing Parallel File-access Patterns on a Large-scale Multiprocessor. In Proceedings of the 9th International Parallel Processing Symposium, pp. , April 1995.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC