N. Nieuwejaar, D. Kotz, A. Purakayastha, C. Ellis, M. Best. File Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10), 1996.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....and allowing parallel file access. Another main problems in parallel I O is efficiently handling byte granularity, non contiguous I O. For instance, parallel scientific applications often access the files non contiguously or their contiguous accesses translates into non contiguous disk accesses[12]. MPI IO [11] and Vesta[3] allow setting linear views on non contiguous file data, whereas Galley Parallel File System[13] offers the user a nested strided interface. Parallel I O access characterization studies [12, 1, 16, 17] have found the poor match between I O access patterns of applications ....

....or their contiguous accesses translates into non contiguous disk accesses[12] MPI IO [11] and Vesta[3] allow setting linear views on non contiguous file data, whereas Galley Parallel File System[13] offers the user a nested strided interface. Parallel I O access characterization studies [12, 1, 16, 17] have found the poor match between I O access patterns of applications and physical layout of data on disks as a large source of I O usage inefficiency. First, a poor match can cause fragmentation of data on the disks of the I O nodes and complex index computations of accesses are needed. Second, ....

[Article contains additional citation context not shown here]

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. Ellis, M. Best. File Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10), 1996.

....In [24] Reddy and Bannerjee studied five scientific applications taken from the PERFECT benchmark suite. They found that most applications exhibited sequential file access patterns. In [23] Bagrodia et al. used Pablo to instrument and characterize out of core scientific applications. In [20], Nieuwejaar et al. performed a multi platform characterization study of file access patterns for parallel scientific workloads as part of the CHARISMA project. I O access patterns were captured on both Intel iPSC 860 and CM5 platforms. Understanding file access patterns is essential when ....

....on a local disk and then there would be no disk contention or communication overhead. Unfortunately, multiple processes may need to read and write the same file space, so proper partitioning must consider file consistency issues. Many characterization studies have targeted I O access patterns [9, 11, 15, 20]. Workload patterns can be recognized both statically (at compile time) or dynamically (at run time) Madhyastha and Reed [15] suggested using learning algorithms to classify I O access patterns at execution time, guiding adaptive file system policies. Memik et al. 17] designed a compiler ....

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. Ellis, and M. Best. File-Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, 1996.

....behavior of the scientific applications has been extensively studied. Cypher et al. 9] studied individual parallel scientific applications, measuring temporal patterns in I O rates. Crandall et al. 8] performed an analysis based on Pablo [1] on three scientific applications. Nieuwejaar et al. [18] characterized a mix of user programs on Intel iPSC and CM 5. All these studies implicitly motivate the usage of MCIO by illustrating that multiple files are accessed within a single run of the studied application. According to these studies, the number of files accessed varies according to the ....

Nieuwejaar, N., Kotz, D., Purakayastha, A., Ellis, C., Best, M. File-Access Characteristics of Parallel Scientific Workloads. In IEEE Transactions on Parallel and Distributed Systems, October 1996 (Vol. 7, No. 10), pp. 1075-1089.

....Parallel applications are in many ways the most similar to pipelined batch applications. The CPU, memory, communication, and I O behavior of parallel and vector applications have been quantified in a number of studies [9, 37, 36] but the most relevant studies consider the impact of explicit I O [30, 32, 23, 8, 1]. Our study embellishes these works by studying the sharing behavior of an important new class of workload. Many of these studies demonstrate the drastic differences in I O behavior for parallel applications compared to general purpose workloads. For example, parallel scientific workloads often ....

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File-access characteristics of parallel scientific workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, 1996.

....necessity of prefetching and caching for maintaining high performance in some parallel I O systems but at the same time show that prefetching and caching, by themselves, are not adequate for maintaining performance with workloads with many clients in this environment. 2.1. 2 CHARISMA Project In [38] the results of the CHARISMA project are described. This was a project designed to characterize the behavior of production parallel workloads at the level of individual reads and writes. The authors traced workloads on two machines, a CM 5 and an iPSC 860, both with numerous scientists running ....

....buffer is equal to the size of the buffer in bytes. An aging system is used to push blocks out to disk eventually if all bytes are not written, effectively falling back to delayed writeback. This scheme is tailored to sequential access patterns, which are common in parallel workloads as seen in [38]. In their tests they use 20 processes, 20 disks, 1K blocks and buffers, and a 4 Mbyte write size. In each test a single file is accessed and a single application run. For many tests they found that caches of 40 80 blocks performed best, with 40 blocks being equivalent to double buffering. ....

[Article contains additional citation context not shown here]

Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, and Michael Best. File-access characteristics of parallel scientific workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, October 1996.

....behavior of the scientific applications has been extensively studied. Cypher et al. 9] studied individual parallel scientific applications, measuring temporal patterns in I O rates. Crandall et al. 8] performed an analysis based on Pablo [1] on three scientific applications. Nieuwejaar et al. [18] characterized a mix of user programs on Intel iPSC and CM 5. All these studies implicitly motivate the usage of MCIO by illustrating that multiple files are accessed within a single run of the studied application. According to these studies, the number of files accessed varies according to the ....

Nieuwejaar, N., Kotz, D., Purakayastha, A., Ellis, C., Best, M. File-Access Characteristics of Parallel Scientific Workloads. In IEEE Transactions on Parallel and Distributed Systems, October 1996 (Vol. 7, No. 10), pp. 1075-1089.

....to the last logical block of each request cannot get a big jump in performance. However, other kinds of file systems such as parallel file systems may produce different results. In many parallel file system traces, for example, the average request size is much smaller (less than 512 bytes) [16], allowing the subsector approach to have a significant improvement in power conservation. Similarly, personal digital assistants and other handheld devices might manage smaller chunks of data, also permitting lower power subblock accesses. An intelligent operating system might further reduce ....

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File-access characteristics of parallel scientific workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, Oct. 1996.

.... or even less) These small requests occur for the following reasons: ffl In many parallel applications (for example, those that access distributed arrays from files) each process needs to access a large number of relatively small pieces of data that are not contiguously located in the file [1, 3, 11, 19, 18, 24]. ffl Most parallel file systems have a Unix like API (application programming interface) that allows a user to access only a single, contiguous chunk of data at a time from a file. 1 Noncontiguous data sets must therefore be accessed by making separate function calls to access each individual ....

....can be defined by using any MPI basic or derived datatype; therefore, any general noncontiguous access pattern can be compactly represented. Several studies have shown that, in many parallel applications, each process needs to access a number of relatively small, noncontiguous portions of a file [1, 3, 11, 19, 24]. From a performance perspective, it is critical that the I O interface can express such an access pattern, as it enables the implementation to optimize the I O request. The optimizations typically allow the physical I O to take place in large, contiguous chunks, even though the user s request may ....

[Article contains additional citation context not shown here]

Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, and Michael Best. File-Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075-- 1089, October 1996.

....and represents a significant obstacle to achieving good performance. The problem is often not with the hardware; many parallel I O subsystems offer excellent performance. Rather, the problem arises from other factors, primarily the I O patterns exhibited by many parallel scientific applications [9, 18, 2, 3, 22, 24, 25, 28]. In particular, each processor tends to make a large number of small I O requests, incurring the high cost of I O on each such request. One reason for this access pattern is that parallel scientific codes frequently involve large arrays distributed across the processor s local memory. After a ....

Nieuwejaar, N., Kotz, D., Purakayastha, A., Ellis, C. and M. Best. File-Access Characteristics of Parallel Scientific Workloads. In IEEE Transactions on Parallel and Distributed Systems, volume 7, number 10, pages 1075--1089, October 1996.

....a selection of logically consecutive disks. The fundamental question, which seems not to have been addressed, is whether we can reduce the update cost per disk, by taking advantage of our knowledge that many consecutive disks are being written. For information on sizes of typical disk writes, see [17], for example. IV 2 Disk operations Our goal is to examine performance, particularly user response time, in a disk array employing a k erasure code. To do this, it is essential to determine how the basic I O operations are completed. Each I O operation maps the logical data to a location ....

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. L. Best. File-access characteristics of parallel scientific workloads. IEEE Trans. Parallel Distrib. Systems 7 (1996), 1075-1089.

....increase parallelism, improve load balancing, and minimize communication overhead [NF95] Each cnode operates on its own piece of data, so from a file system s point of view, each cnode makes its own requests to read or write data. However, this data is usually not logically contiguous in the file [NKP96], hence a separate file system request must be made for each contiguous portion. Consequently, the file system receives numerous, concurrent small requests from many processors instead of a single, large request that would have occurred on a uniprocessor. Additionally, since typical parallel file ....

....available ionodes, each request made from the application may be further broken down into smaller requests, which are finally sent to different ionodes. The problem here is that good data distributions for the application tends to lead to small data blocks and non sequential access to the data [NKP96]. A straight forward implementation of these I O requests using current parallel I O interfaces leads to uncoordinated accesses among the cnodes. This results in abysmal performance [NF95] Two important pieces of information has been lost 9 in the process [Kot97] 1) each cnode s request is ....

Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, and Michael Best. "File-Access Characteristics of Parallel Scientific Workloads." IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, October 1996.

No context found.

Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, and Michael Best. File-access characteristics of parallel scientific workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, October 1996.

No context found.

Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, and Michael Best. File-access characteristics of parallel scientific workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, October 1996.

No context found.

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. Ellis, M. Best. File Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10), 1996.

No context found.

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. Ellis, and M. Best. File Access Characteristics of Parallel Scientific Workloads. In IEEE Transactions on Parallel and Distributed Systems, 7(10), Oct. 1996.

No context found.

Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, Michael L. Best. File Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10), October 1996.

No context found.

N. Nienwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File access characteristics of parallel scientific workloads. IEEE Trans. Parallel and Distributed Systems, 7(10):1075--1088, 1996.

No context found.

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File-Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, 1996.

No context found.

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File-Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, 1996.

No context found.

Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, and Michael L. Best. Fileaccess characteristics of parallel scientific workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, October 1996.

No context found.

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File-Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, 1996.

No context found.

N. Nieuwejaar, D. Kotz, A. Purakayastha, C.S. Ellis, and M. Best, File-Access Characteristics of Parallel Scientific Workloads, IEEE Trans. Parallel and Distributed Systems, vol. 7, no. 10, pp. 10751089, Oct. 1996.

No context found.

N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best. File-access characteristics of parallel scientific workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10):1075--1089, Oct. 1996.

No context found.

N Nieuwejaar, D Kotz, A Purakayastha, C Ellis, M Best. "File-Access Characteristics of Parallel Scientific Work loads". IEEE Tran. Par. and Dist. Sys., 7(10), Oct 1996.

No context found.

Nils Nieuwejaar, David Kotz, Apratim Purakayastha, Carla Schlatter Ellis, Michael L. Best. File Access Characteristics of Parallel Scientific Workloads. IEEE Transactions on Parallel and Distributed Systems, 7(10), October 1996.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC