algorithm for load balancing. The following sections elaborate on each step in the above algorithm, presenting various design decisions that one encounters. 2.1 Load Evaluation The efficacy of any load balancing scheme is directly dependent on the quality of load evaluation. Good load measurement is necessary both to determine that a load imbalance exists and to calculate how much work should be transferred to alleviate that imbalance. One can determine the load associated with a given task analytically, empirically or by a combination of those two methods. 6 CHAPTER 2. METHODOLOGY 2.1.1 Analytic Load Evaluation The load for a task is estimated based on knowledge of the time complexity of the algorithm(s) that task is executing along with the data structures on which it is operating. For example, if one knew that a task involved merge sorting a list of 64 elements, one might estimate the load to be 384, since merge sort is an O(N log 2 N) sorting algorithm, and since 64 log 2 (64) ...

This paper describes basic programming technology to support irregular applications on scalable concurrent hardware and shows how the technology has been applied to a variety of large-scale industrial application problems. The technology is based on the concept of a concurrent graph library that provides an adaptive collection of light-weight threads that may relocate between computers dynamically. The graph is portable to a wide range of highperformance multicomputers, shared-memory multiprocessors, and networked workstations. For each machine it is optimized to take advantage of the best available underlying communication and synchronization mechanisms. The graph provides a framework for adaptive refinement of computations, automatic load balancing, and interactive, on-the-fly visualization. It has been applied to a variety of large scale irregular applications to provide portable, scalable implementations with substantial code reuse. The applications described in this article typify...

Parallel computations comprised of multiple, tightly interwoven phases of computation may require a different approach to dynamic load balancing than single-phase computations. This paper presents a load sharing method based on the view of load as a vector, rather than as a scalar. This approach allows multiphase computations to achieve higher efficiency on large-scale multicomputers than possible with traditional techniques. Results are presented for two large-scale particle simulations running on 128 nodes of an Intel Paragon and on 256 processors of a Cray T3D, respectively. INTRODUCTION Load balancing techniques already in the literature have concentrated entirely on single-phase computations (Boillat 1990; Cybenko 1989; Evans and Butt 1993; Heirich and Taylor 1995; Horton 1993; Kohring 1995; Lin and Keller 1987; Muniz and Zaluska 1995; Song 1994; Walshaw and Berzins 1995; Watts et al. 1996; Willebeek-LeMair and Reeves 1993; Williams 1991; Xu and Lau 1997). That is, they work only ...

. In this paper, we present a cohesive, practical load balancing framework that addresses many shortcomings of existing strategies. These techniques are portable to a broad range of prevalent architectures, including massively parallel machines such as the Cray T3D and Intel Paragon, shared memory systems such as the SGI Power Challenge, and networks of workstations. This scheme improves on earlier work in this area and can be analyzed using well-understood techniques. The algorithm operates using nearest-neighbor communication and inherently maintains existing locality in the application. A simple software interface allows the programmer to use load balancing with very little effort. Unlike many previous efforts in this arena, the techniques have been applied to large-scale industrial applications, one of which is described herein. 1 Introduction A number of trends in computational science and engineering have increased the need for effective dynamic load balancing techniques. In par...

This paper presents techniques for dynamic load balancing in heterogeneous computing environments. That is, the techniques are designed for sets of machines with varying processing capabilities and memory capacities. These methods can also be applied to homogenous systems in which the effective compute speed or memory availability is reduced by the presence of other programs running outside the target computation. To handle heterogeneous systems, a precise distinction is made between an abstract quantity of work, which might be measured as the number of iterations of a loop or the count of some data structure, and the utilization of resources, measured in seconds of processor time or bytes of memory, required by that work. Once that distinction is clearly drawn, the modifications to existing load balancing techniques are fairly straight-forward. The effectiveness of the resulting load balancing system is demonstrated for a large-scale particle simulation on a network of heterogeneous P...

Traditional load balancing techniques measure the utilization of computers as scalars. This one-dimensional view fails to capture the more complex behavior of certain classes of concurrent computations. To resolve this shortcoming, we present the view of resource utilization as a vector, allowing multiple kinds of resources to be simultaneously reassigned. Applications that might benefit from this approach include those comprised of multiple computational phases, those with disparate processor and memory requirements, and those with loads changing in a rapid, but predictable fashion. Results are given for both synthetic and real applications running on smallto large-scale multicomputers and networks of workstations. Keywords: dynamic resource allocation, dynamic load balancing, irregular applications 1 Introduction Dynamic load balancing techniques currently in the literature characterize resource utilization by a single number. While such a representation is adequate for many appli...