Remzi H. Arpaci-Dusseau, Eric An derson , Noah Treuhaft, David E. Culler, Joseph M. Hellerstein , David Patterson , an d Kathy Yelick. Cluster I/O with River:Makin g the fast casecommon . Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems (IOPADS '99), pages 10--22, Atlan ta, Georgia, May 1999.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....rely on caching common subexpressions [25, 30, 33, 37] Nevertheless, deploying these techniques in a broader context, specifically for data analysis applications, remains a challenging problem. Several database and middleware frameworks that target this class of applications have been developed [6, 7, 8, 13, 20]. Although these frameworks provide efficient and scalable common runtime support for a wide range of applications, they do not attempt to leverage inter and intra query commonalities when executing multiple query workloads. When data analysis techniques are employed in a collaborative ....

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. Patterson, and K. Yelick. Cluster I/O with River: Making the fast case common. In Proceedings of the Sixth Workshop on I/O and Parallel and Distributed Systems, Atlanta, GA, 1999.

....network appliances) In addition, even though Scout paths can be optimized by reducing the number of stages, run time recon guration of path functionality is not supported. The use of explicit data ows as a means for expressing and organizing computations in a cluster has been examined in River [15] and Tigris [16] a Java implementation of River) Our data access path are logically at a lower level, and we are dealing with the recon guration and functional composition of storage device access instead of computation. The concept of a Proboscis resembles the derived virtual devices [17] ....

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. Patterson, and K. Yelick, \Cluster I/O with river: Making the fast case common," in Proceedings of IOPADS '99, 1999.

....matter of attaching an application to its storage. Distributed systems are prone to performance variations, failed connections, and exhausted resources. These problems cannot be solved merely by increasing hardware capacity or reliability. They are often integral properties of distributed hardware [6], opportunistic resources [21] and social scheduling constraints. Grid applications are not prepared to deal with any of these conditions. Often designed to run in the relatively predictable environment of a standalone machine, they expect low latency, reliable delivery, and unlimited storage. ....

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. Patterson, and K. Yelick. Cluster I/O with River: Making the fast case common. In Proceedings of IOPADS, May 1999.

....the problems raised by ad hoc and adaptive applications. Many systems have been built to enable applicationspeci c composition. Some of the more successful examples, including the Click modular router and the Rivers system for performing database queries, already follow many of our guidelines [14, 4]. Though our framework is obviously not appropriate for all domains of applications, we have attempted to design a more general purpose composition framework. Tuple spaces follow many of our principles, but because they do not explicitly support composition of services, we do not consider tuple ....

Remzi H. Arpaci-Dusseau, Eric Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstein, David Patterson, and Kathy Yelick. Cluster I/O with River: Making the Fast Case Common.

....Druschel and Banga demonstrate that with web servers running on traditional interrupt driven operating systems, a slight increase in load beyond the capacity of the server can drive the server into a persistent state of livelock, drastically reducing its e ective throughput. As a third example, in [1], Arpaci Dusseau et al. demonstrate that with conventional software architectures, the di erence in performance resulting from placing data on the inner tracks vs. outer tracks of a single disk can a ect the global throughput of an eight node cluster of workstations by up to 50 . By their very ....

Remzi H. Arpaci-Dusseau, Eric Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstein, David A. Patterson, and Katherine Yelick. Cluster I/O with River: Making the Fast Case Common. In Proceedings of the Sixth Workshop on I/O in Parallel and Distributed Systems (IOPADS '99), May 1999.

....with. There are some important classes of data for which it is impossible to gather accurate statistics ahead of time, as with live stock market feeds or measurements of the San Andreas fault activity. We present a continuously re optimizing query processing system (CRQP) built in the River [1] programming environment, which performs no query optimization before running a query. During execution, the CRQP optimizes query execution on a tuple by tuple basis, selecting query plans dynamically, based upon the current query execution situation. The CRQP organizes each query as a set of ....

....new tuple is thus the first operator in that tuple s query plan. This model does not require an initial plan and therefore no optimizer is used. Instead, the CRQP reoptimizes towards the best query plan during execution. 3. 2 IMPLEMENTATION We implemented the CRQP, shown in Figure 3 in the River [1] programming environment for clusters of computers. River applications are constructed as modules with one or more input and output channels. River modules acquire data from upstream , process it, and put it in their output channels to flow downstream . River distributed queues (DQ) connect data ....

Remzi H. Arpaci-Dusseau, Eric Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstein, David Patterson, and Katherine Yelick. Cluster i/o with river: Making the fast case common. In Input/Output for Parallel and Distributed Systems (IOPADS), Atlanta, Georgia, May 1999.

....the processors in group G i . This is to measure the contention for the shared file system in an SMP. The file system results are merged and stored in Thruputs[ Thruputs[k] fs has the file system throughput per processor for the (Thruputs[k] nid) th SMP when Thruputs[k] procs processors (G i [1]; G i [j] are writing, and Thruputs[k] pid is the (Thruputs[k] procs)th processor ID in the given SMP. The ratio between the I O throughputs obtained by the fastest and the slowest processor in I O is returned and that is used as an indication of the degree of heterogeneity in I O. The ....

....placement of I O servers could be added to PIOUS for heterogeneous environments. VIP FS [10] and VIPIOS [3] consider network congestions or data locality, as does our work in [6] Our heuristic selection and placement of I O servers could be used with VIP FS and VIPIOS in distributed environments. [1] describes river, a new data flow programming environment and I O substrate for cluster environments. It is similar to our work in that I O load balancing is done internally without user intervention. However, since [1] adopts their own new data flow programming paradigm, existing parallel ....

[Article contains additional citation context not shown here]

R. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. Culler, J. Hellerstein, D. Patterson, and K. Yelick. Cluster I/O with River: Making the Fast Case Common. In Proceedings of the Sixth Workshop on I/O in Parallel and Distributed Systems, Atlanta, Georgia, 1999.

....configurations and heterogenous clusters as the one being built at UCSD, which as described above presents the interesting problem of balancing the work in such a way that every node contributes to the solution according to its resources. There are severl interesting projects exploring this issue[4], but they are limited to overall good overall performance, not the best performance. 51 Appendix A Kayak XU Internals Here we described the Kayak XU internals making special emphasis on those components that play an important role in an input output operation. This NCSA NT Supercluster was an ....

Remzi H. Arpaci, Erik Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstin, David A. Patterson, and Kathy Yelick. Cluster i/o with river: Making the fast case common. available from http://www.cs.berkeley.edu/ remzi/papers.html.

....and per object resource consumption. All of these systems use long term histories to make good installation time or invocationtime function placement decisions. ABACUS complements these previous systems by looking at how to dynamically adapt placement decisions to run time conditions. River [4] is a system that dynamically adjusts per consumer rates to match production rates, and per producer rates to meet consumption rate variations. Such adjustments allow it to adapt to run time non uniformities among cluster systems performing the same task. ABACUS complements River by adapting ....

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. Patterson, and K. Yelick. Cluster I/O with River: Making the fast case common. In Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pages 10--22, Atlanta, GA, May 1999.

....parallel supercomputing environment. These clusters have been used for a wealth of di erent kinds of applications, ranging from scienti c computing [69] backbones for high throughput, shared computing environments [25, 39, 48] glori ed batch computing systems [56] parallel I O environments [8, 11], and platforms for building scalable, fault tolerant, highly available infrastructural Internet services [20, 34, 35] In this work, we focus on the use of clusters for building infrastructural Internet services. Modern Internet services necessarily possess similar properties to traditional ....

....it routes requests to the least loaded of equivalent software components in order to perform load balancing. However, TACC assumes that all shared state in the system can be reconstructed if lost and does not need to be kept persistent or consistent. Similarly, the Rivers I O processing platform [11] implicitly load balances across I O multiplexing operators by allowing them to pull data towards themselves at their own speed, but Rivers does not deal with data persistence or fault tolerance. Fundamentally, none of these cluster toolkits provide support or abstractions for fault tolerant, ....

[Article contains additional citation context not shown here]

Remzi H. Arpaci-Dusseau, Eric Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstein, David A. Patterson, and Katherine Yelick. Cluster I/O with River: Making the Fast Case Common. In Proceedings of the Sixth Workshop on I/O in Parallel and Distributed Systems (IOPADS '99), May 1999.

....in Java. The goal of Tigris is to facilitate development of applications which can dynamically utilize workstation cluster resources for high performance computing and I O, by automatically balancing resource load across the cluster as a whole. Tigris borrows many of its concepts from River [3], a cluster I O system implemented on C on the Berkeley Network of Workstations [24] By exploting the use of Java as the native execution and control environment in Tigris, we believe that cluster application development is greatly simpli ed, and that applications can take advantage of code ....

....machine code segments which perform native operations, such as low overhead communication. 2 The Tigris System Modules Disks Distributed Queue Modules Disks Modules Read Data Write Data Sort, etc. Hash Join, Figure 1: A sample River application. The key ideas in Tigris are borrowed from River [3], a system supporting cluster based applications which automatically balance CPU, network, and disk I O load across the cluster as a whole. River employs a data ow programming model wherein applications are expressed as a series of modules each supporting a very simple input output interface. ....

[Article contains additional citation context not shown here]

R. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. Culler, J. Hellerstein, D. Patterson, and K. Yelick. Cluster i/o with river: Making the fast case common. In IOPADS '99, 1999. http://www.cs.berkeley.edu/~remzi /Postscript/river.ps.

....As a result, dataflow based models have been proposed by several researchers. Barclay et al. [11] proposed a dataflow based technique for parallelizing the loading of a large database. Similar techniques are used by the Gamma [17] and Volcano [20] parallel databases. Recently, Arpaci Dusseau et al. [8] have proposed a dataflow based programming model for scheduling I O intensive tasks on clusters. 8 Conclusions There are four main conclusions of our study. First, for the same I O interconnect, disks, and number of processors, Active Disk configurations perform better than the corresponding SMP ....

R. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. Culler, J. Hellerstein, D. Patterson, and K. Yelick. Cluster I/O with Rivers: Making the Fast Case Common. Submitted for publication, 1998.

.... in large scale systems, and that system support for building robust programs is needed [5] Thus, we began work on River, a programming environment that provides mechanisms to enable consistent and high performance in spite of erratic performance in underlying components, focusing mainly on disks [7]. However, River itself does not handle absolute correctness faults in an integrated fashion, relying either upon retry after failure or a checkpointrestart package. River also requires applications to be completely rewritten to enable performance robustness, which may not be appropriate in many ....

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. A. Patterson, and K. Yelick. Cluster I/O with River: Making the Fast Case Common. In IOPADS '99, May 1999.

....that GALE re organizes or replicates the data, taking the current climate into account. SToRM must be able to adaptively take advantage of replicated sources of data under reads. Our earlier work on Graduated Declustering focused on the distributed, adaptive use of mirrors for parallel clients [5]; we plan to generalize it to handle more general purpose workloads and a variety of replicated layout schemes. 2.2 GALE Short term adaptation does not solve all of the problems encountered in dynamic, heterogeneous environments. Short term adaptations are analogous to greedy algorithms, which ....

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. A. Patterson, and K. Yelick. Cluster I/O with River: Making the Fast Case Common. In IOPADS '99, May 1999.

....into the system to query remote entities about their state and behavior. To investigate the value of our approach and taxonomy, we perform an in depth exploration of the trade offs of the different sources of information for a particular distributed system: the River I O environment for clusters [4, 5]. The main goal of River is to enable data intensive applications to adapt to performance faults of components in a cluster that is, to perform well in spite of unexpected variations in performance of disks, workstations, or other components within the system. Central to run time adaptation in ....

....we give a brief summary of our application experience within River. Once we had a solid information centric understanding of each algorithm, the final implementation of both mechanisms was straight forward. The result is better performance for applications as compared to previous efforts [5]. 5 We present the performance of six I O intensive, database query processing primitives, each of which has been transformed from a static parallel application into a robust and adaptive version. Programmers insert distributed queues and graduated declustering into their applications to form ....

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. A. Patterson, and K. Yelick. Cluster I/O with River: Making the Fast Case Common. In IOPADS '99, May 1999.

....database that uses a hash on the primary index of a database table to statically partition the table across cluster nodes. This data placement allows data parallelism and improves the load balance. In contrast, Hippodrome dynamically reassigns stores based on observed device utilizations. River [8] is a cluster based I O architecture that uses credit based back pressure and graduated declustering (GD) to distribute work in a manner proportional to the speed of the recipient nodes. However, River requires modifying the application, and it makes shortterm load balancing decisions, and does ....

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. Patterson, and K. Yelick. Cluster I/O with River: Making the fast case common. In 6th Workshop on Input/Output in Parallel and Distributed Systems (IOPADS'99), pages 10--22, Atlanta, GA, May 1999.

....of a mirrored file on multiple storage nodes, to tolerate failures up to the replication degree. Mirroring consumes more storage and network bandwidth than striping with parity, but it is simple and reliable, avoids the overhead of computing and updating parity, and allows load balanced reads [5, 16]. 3.2 Name Space Operations Effectively distributing name space requests presents different challenges from I O request routing. Name operations involve more computation, and name entries may benefit more from caching because they tend to be relatively small and fragmented. Moreover, directories ....

Remzi H. Arpaci-Dusseau, Eric Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstein, David A. Patterson, and Katherine Yelick. Cluster I/O with River: Making the fast case com- mon. In I/O in Parallel and Distributed Systems (IOPADS), May 1999.

....and operate read only to answer client queries. Future work will look at more elaborate ways of placing data and dynamically migrating data in response to access patterns. e.g. we d like to incorporate some ideas from the River system which handles run time load perturbations in write workloads [Arpaci Dusseau et al. 1999]. The number of scanner nodes used by an application can range from one to the total number of nodes in the system. The appropriate number depends on what part of the system is the bottleneck. When the storage nodes don t have enough processing power for the scanners to keep up with their disks ....

Arpaci-Dusseau, R. H., Anderson, E., Treuhaft, N., Culler, D. E., Hellerstein, J. M., Patterson, D., and Yelick, K. Cluster I/O with river: Making the fast case common. In Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems (IOPADS-99), pp. 10--22, Atlanta, Georgia, May 5, 1999.

No context found.

R. H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. E. Culler, J. M. Hellerstein, D. A. Patterson, and K. Yelick. Cluster I/O with River: Making the Fast Case Common. In Sixth Workshop on I/O in Parallel and Distributed Systems (IOPADS '99), pages 10--22, Atlanta, May 1999.

....use of Java as the native execution and control environment in Tigris, we believe that cluster application development is greatly simplified and that applications can take advantage of code mobility, strong typing, and other features provided by Java. The key ideas in Tigris build upon those River [5], a system which was implemented in C on the Berkeley Network of Workstations [25] We describe the major di#erences between Tigris and River in Section 6. Tigris achieves high performance communication and disk I O through the use of Jaguar [28] an extension of the Java programming ....

....traditional Datamation sort implementations, the cost of compiling the application and cold booting the operating system are not measured. 6 Related Work Tigris relates most closely to work in the areas of cluster programming frameworks and parallel databases. Tigris design is based on River [5], a robust cluster based I O system implemented in C on the Berkeley Network of Workstations [25] While facially quite similar, Tigris di#ers from the original River system in a number of important respects, mainly stemming from the use of Java and Jaguar. In River, communication channels ....

R. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D. Culler, J. Hellerstein, D. Patterson, and K. Yelick. Cluster I/O with River: Making the fast case common. In IOPADS '99, 1999. http://www.cs.berkeley.edu/~remzi/Postscript/river. ps.

No context found.

Remzi H. Arpaci-Dusseau, Eric Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstein, David A. Patterson, and Katherine Yelick. Cluster I/O with River: Making the Fast Case Common. In Sixth Workshop on I/O in Parallel and Distributed Systems (IOPADS '99), pages 10--22, Atlanta, May 1999.

No context found.

Remzi H. Arpaci-Dusseau, Eric An derson , Noah Treuhaft, David E. Culler, Joseph M. Hellerstein , David Patterson , an d Kathy Yelick. Cluster I/O with River:Makin g the fast casecommon . Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems (IOPADS '99), pages 10--22, Atlan ta, Georgia, May 1999.

No context found.

Remzi H. Arpaci-Dusseau, Eric An derson , Noah Treuhaft, David E. Culler, Joseph M. Hellerstein , David Patterson , an d Kathy Yelick. Cluster I/O with River:Makin g the fast casecommon . Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems (IOPADS '99), pages 10--22, Atlan ta, Georgia, May 1999.

No context found.

Remzi H. Arpaci-Dusseau, Eric Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstein, David A. Patterson, Katherine A. Yelick: Cluster I/O with River: Making the Fast Case Common. IOPADS 1999: 10-22

No context found.

Remzi H. Arpaci-Dusseau, Eric Anderson, Noah Treuhaft, David E. Culler, Joseph M. Hellerstein, David Patterson, and Kathy Yelick. Cluster I/O with River: Making the fast case common. In Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pages 10--22, Atlanta, GA, May 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC