Applications based upon the finite element method are well known for their demand for computational resources. An effective method for satisfying this demand is heterogeneous parallel computing. This paper presents the results obtained by applying heterogeneous computing to a large, existing finite element application code: CSTEM. A difficult problem associated with heterogeneous computing is the mapping and scheduling problem---the process of assigning the tasks of a parallel program to the individual processors. A simple assignment heuristic, Levelized Min-Time (LMT), is presented, along with simulated results from applying the LMT algorithm to heterogeneous CSTEM on a variety of different heterogeneous machine clusters.

Utilizing a collection of workstations and supercomputers in a metacomputing environment does not only offer an enormous amount of computing power, but also raises new problems. The true potential of WAN-based distributed computing can only be exploited if the application-to-architecture mapping reflects the different processor speeds, network performances and the application's communication characteristics. In this paper, we present the Metacomputer Adaptive Runtime System MARS,a framework for minimizing the execution time of distributed applications on a WAN metacomputer. Work-load balancing and task migration is based on dynamic information on the processor load and network performance. Moreover, MARS uses accumulated statistical data on previous execution runs of the same application to derive an improved task-to-process mapping. Migration decisions are based on (1) the current system load, (2) the network load and (3) previously obtained application-specific characteri...

Given a distributed multimedia database system and a set of queries as well as their frequencies from each site, the objective of a data allocation algorithm is to locate the multimedia data objects (MDOs) at different sites so as to minimize the total data transfer cost incurred in executing the queries. The data allocation problem, however, is NP-complete, and thus requires fast heuristics to generate efficient solutions. In this paper we propose three data allocation algorithms which are based on a genetic technique, an evolutionary process, and neural networks. We have implemented and evaluated these algorithms on our distributed multimedia database system test-bed. A comparison of the algorithms reveals trade-offs between their solution quality and time-complexity. 1

A metasystem is a shared ensemble of workstations, vector, and parallel machines

Distributed systems comprising networked heterogeneous workstations are now considered to be a viable choice for high-performance computing. For achieving a fast response time from such systems, an eficient assignment of the application tasks to the processors is imperative. The general assignment problem is known to be NP-hard, except in a few special cases with strict assumptions. While a large number of heuristic techniques have been suggested in the literature that can yield sub-optimal solutions in a reasonable amount of time, we aim to develop techniques for optimal solutions under relaxed assumptions. The basis of our research is a best-Jrst search technique known as the A * algorithm from the area of artijicial intelligence. The original search technique guarantees an optimal solution but is not feasible for problems of practically large sizes due to its high time and space complexity. We propose a number of algorithms based around the A * technique. The proposed algorithms also yield optimal solutions but are considerably fastel: The first algorithm solves the assignment problem by using parallel processing. Parallelizing the assignment algorithm is a natural way to lower the time complexity, and we believe our algorithm to be novel in this regard. The second algorithm is based on a clustering based pre-processing technique that merges the high afinity tasks. Clustering reduces the problem size, which in tum reduces the state-space for the assignment algorithm. We also propose three heuristics which do not guarantee optimal solutions but provide near-optimal solutions and are considerably fastel: By using our parallel formulation, the proposed clustering technique and the heuristics can also be parallelized to further improve their time complexity. Keywords: Best-first search, parallel processing, task assignment. mapping, distributed systems.

This research investigates the problem of allocating a set of heterogeneous applications to a set of heterogeneous machines connected together by a high-speed network. The proposed resource allocation heuristics were implemented on the High Performance Distributed Computing Program's (HiPer-D) Naval Surface Warfare Center testbed. The goal of this study is to design static resource allocation heuristics that balance the utilization of the computation and network resources while ensuring very low failure rates. A failure occurs if no allocation is found that allows the system to meet its resource and quality of service constraints. The broader goal is to determine an initial resource allocation that maximizes the time before run-time re-allocation is required for managing an increased workload. This study proposes two heuristics that perform well with respect to the loadbalancing and failure rates. These heuristics are, therefore, very desirable for HiPer-D like systems where low failure rates can be a critical requirement.

This paper presents techniques for dynamic load balancing in heterogeneous computing environments. That is, the techniques are designed for sets of machines with varying processing capabilities and memory capacities. These methods can also be applied to homogenous systems in which the effective compute speed or memory availability is reduced by the presence of other programs running outside the target computation. To handle heterogeneous systems, a precise distinction is made between an abstract quantity of work, which might be measured as the number of iterations of a loop or the count of some data structure, and the utilization of resources, measured in seconds of processor time or bytes of memory, required by that work. Once that distinction is clearly drawn, the modifications to existing load balancing techniques are fairly straight-forward. The effectiveness of the resulting load balancing system is demonstrated for a large-scale particle simulation on a network of heterogeneous P...

.............................................................................................................................................. xix Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 1.1 Overview.................................................................................................................................. 1 1.2 Parallel Architectures and The Scheduling Problem......................................................... 3 1.3 Research Objectives ................................................................................................................ 5 1.4 Contributions........................................................................................................................... 6 1.5 Organization of the Thesis .................................................................................................... 7 Chapter 2 Background and Literature Survey . . . . . . . . . . . . . . ...

..............................................................................................................................................xiii Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 1.1 Overview.................................................................................................................................. 1 1.2 Parallel Architectures and the Scheduling Problem.......................................................... 3 1.3 A Taxonomy of Approaches to the Scheduling Problem ................................................. 5 1.4 Outline of the Thesis .............................................................................................................. 7 Chapter 2 Evolution of the Scheduling Problem . . . . . . . . . . . . . . . . . . . .8 2.1 Introduction ............................................................................................................................. 8 2.2 Prob...

While metacomputers o#er an enormous amount of computing power the performance of the single computing nodes and the interconnection network mayvary at program runtime. Dynamical task migration is needed to fully utilize the available resources. In this paper, we presentascheme for deriving optimal task migration decisions. The decisions are based on # long-term statistics on the network and CPU performance, # previously acquired information on the program's runtime behavior, # the currentavailability and performance of computing nodes and network. The information is obtained at runtime without signi#cantly increasing the application's execution speed. Information from previous execution runs is used for improving future migration decisions. By combining statistical data of several runs, characteristic patterns in the program's execution #ow are determined, which can be used to improve the overall execution behavior. Our current prototype implementation is based on MPI, but the general scheme is applicable to any programming model. 1 1