In this paper we analyze the I/O access patterns of a widely-used biological sequence search tool and implement two variations that employ parallel-I/O for data access based on PVFS (Parallel Virtual File System) and CEFT-PVFS (Cost-Effective Fault-Tolerant PVFS). Experiments show that the two variations outperform the original tool when equal or even fewer storage devices are used in the former. It is also found that although the performance of the two variations improves consistently when initially increasing the number of servers, this performance gain from parallel I/O becomes insignificant with further increase in server number. We examine the effectiveness of two read performance optimization techniques in CEFT-PVFS by using this tool as a benchmark. Performance results indicate: (1) Doubling the degree of parallelism boosts the read performance to approach that of PVFS; (2) Skipping hotspots can substantially improve the I/O performance when the load on data servers is highly imbalanced. The I/O resource contention due to the sharing of server nodes by multiple applications in a cluster has been shown to degrade the performance of the original tool and the variation based on PVFS by up to 10 and 21 folds, respectively; whereas, the variation based on CEFT-PVFS only suffered a two-fold performance degradation. Keywords: parallel I/O, CEFT-PVFS, PVFS, BLAST 1.

... In this paper, we present a formal model of the behavior of CPU and I/O interactions in scientific applications, from which we derive various formulas that characterize application performance. Our model captures the I/O and CPU activity at different levels of granularity, where results from the model are shown to be in excellent agreement with measurement data from a set of I/O-intensive applications. Using the formulas from our model, which explicitly take I/O activity into account, we also present examples of possible applications of the model.

Scientific applications are increasingly being implemented on massively parallel supercomputers. Many of these applications have intense I/O demands, as well as massive computational requirements. This paper is essentially an annotated bibliography of papers and other sources of information about scientific applications using parallel I/O. It will be updated periodically.

Despite continued innovations in disk design, input/output performance has not kept pace with concurrent increases in processor speeds. Much research has focused on developing algorithms to avoid input/output or hide input/output latency in an attempt to redress this widening gap. Many studies have shown that with advance knowledge of access patterns, file systems can improve input/output performance by selecting policies appropriate for the resource demands. Unfortunately, access patterns may be complex or data dependent, and therefore unknown a priori. Our thesis is that the file system can automatically detect qualitative file access patterns both locally (per parallel program thread) and globally (per parallel program) and use this information to dynamically choose appropriate file system policies. We propose two complementary methods for automatic classification, based on neural networks and hidden Markov models, respectively. Global classifications are created from a combination...

This paper presents the integration of two collective I/O techniques into the Clusterfile parallel file system : diskdirected I/O and two-phase I/O. We show that global cooperative cache management improves the collective I/O performance. The solution focuses on integrating disk parallelism with other types of parallelism: memory (by bu#ering and caching on several nodes), network (by parallel I/O scheduling strategies) and processors (by redistributing the I/O related computation over several nodes). The performance results show considerable throughput increases over ROMIO's extended two-phase I/O.

As disk capacities continue to rise more rapidly than transfer rates, adaptive, redundant striping smoothly trades capacity for higher performance. We developed a fuzzy logic rule base for adaptive, redundant striping of les across multiple disks. This rule base is based on a queuing model of disk contention that includes le request sizes and disk hardware parameters. At low loads, the rule base stripes aggressively to minimize response time. As loads rise, it stripes less aggressively to maximize aggregate throughput. This adaptive striping rule base is incorporated into our second generation Portable Parallel File System (PPFS II). Experimental results showed that the analytical models of disk striping are capable of accurately predicting le system behavior. Also, it is shown that, depending on the access pattern, adaptive striping can double the input/output performance compared to striping with xed distribution parameters. 1 Introduction As new high-performance computing sys...

Abstract — This paper introduces a new benchmark tool,

This paper was the starting point of MPI-IO in that it was the first attempt to exploit this analogy by applying the (then relatively new) MPI concepts for message passing to the realm of parallel I/O.

With increasing development of applications for heterogeneous, distributed computing grids, the focus of performance analysis has shiftedfrom a posteriori optimization on homogeneous parallel systems to application tuning for heterogeneous resources with time varying availability. This shift has profound implications for performance instrumentation and analysis techniques. Autopilot is a new infrastructure for dynamic performance tuning of heterogeneous computational grids based on closedloop control. This paper describes the Autopilot model of distributed sensors, actuators, and decision procedures, reports preliminary performancebenchmarks, and presents a case study in which the Autopilot library is utilized in the development of an adaptive parallel input/output system.

This paper presents the design and implementation of the MPI-IO interface for the Clusterfile parallel file system. The approach offers the opportunity of achieving a high corelation between the file access patterns of parallel applications and the physical file distribution. First, any physical file distribution can be expressed by means of MPI data types. Second, mechanisms such as views and collective I/O operations are portably implemented inside the file system, unifying the I/O scheduling strategies of the MPI-IO library and the file system. The experimental section demonstrates performance benefits of more than one order of magnitude. 1