The resource availability in Grids is generally unpredictable due to the autonomous and shared nature of the Grid resources and stochastic nature of the workload resulting in a best effort quality of service. The resource providers optimize for throughput and utilization whereas the users optimize for application performance. We present a cost-based model where the providers advertise resource availability to the user community. We also present a multi-objective genetic algorithm formulation for selecting the set of resources to be provisioned that optimizes the application performance while minimizing the resource costs. We use trace-based simulations to compare the application performance and cost using the provisioned and the best effort approach with a number of artificially generated workflow-structured applications and a seismic hazard application from the earthquake science community. The provisioned approach shows promising results when the resources are under high utilization and/or the applications have significant resource requirements.

Abstract — In order to fulfill the complex resource requirements of some users in Grid environments, support for coallocation between different resource providers is needed. Here, it is quite difficult to coordinate these different services from different resource providers, because a Grid scheduler has to cope with different policies and objectives of the different resource providers and of the users. Agreement-based resource management is considered a feasible solution to solve many of these problems as it supports the reliable interaction between different providers and users. However, most current models do not well support co-allocation. Here, negotiation is needed to create such bi-lateral agreements between several Grid parties. Such a negotiation process should be automated with no or minimal human interaction, considering the potential scale of Grid systems and the amount of necessary transactions. Therefore, strategic negotiation models play an important role. In this paper, a negotiation models which supports the co-allocation between different resource providers are proposed and examined. First simulations have been conducted to evaluate the presented system. The results demonstrate that the proposed negotiation model are suitable and effective for Grid environments. I.

Distributed High-Performance Computing Applications are commonplace in several scientific domains. To correctly execute such applications requires the allocation of multiple resources; often, these resources will belong to multiple administrative domains, and so will not be under centralized control. Typically, dedicated resources of different types will be required, usually including compute resources and network bandwidth. This paper presents HARC (Highly-Available Robust Co-scheduler): a co-scheduling framework suitable for any resource under the control of a scheduler supporting reservations. At the core of HARC is a replicated co-scheduler process, which gives the system fault-tolerance. Lamport’s Paxos Consensus algorithm is used both to ensure consistency, and to ensure progress provided a majority of the replicas continue to function. It is shown that a deployment of seven replicas can be expected to have a Mean-Time-To-Failure measured in years. The design of HARC is presented in detail; specifically, HARC is shown to be well suited to the scheduling of large scientific workflows. Experiences with the first implementation of HARC are presented. This includes the successful use of HARC to co-schedule reservations on ten compute nodes and two Calient Diamondwave optical network switches as part of a demo at iGrid 2005. 1.

One of the promises of Grid Computing is to enable the execution of applications across multiple sites. Several multi-site applications require simultaneous access to resources hosted on autonomous domains; this problem is known as resource co-allocation. Projects working on resource co-allocation face four major problems: distributed transactions, fault tolerance, intersite network overhead, and schedule optimization. Although resource co-allocation is fundamental for Grid Computing, no survey has covered the current projects, solutions, and open challenges on this topic. Therefore, in this chapter, we describe the challenges on resource coallocation, present the projects developed over the last decade, and classify them according to their similar characteristics. In addition, we discuss open research issues and trends such as negotiation, advance reservations, and rescheduling of multi-site applications.

Abstract not found

Workflows are widely used in applications that require coordinated use of computational resources. Workflow definition languages typically abstract over some aspects of the way in which a workflow is to be executed, such as the level of parallelism to be used or the physical resources to be deployed. As a result, a workflow management system has the responsibility of establishing how best to map tasks within a workflow to the available resources. As workflows are typically run over shared resources, and thus face unpredictable and changing resource capabilities, there may be benefit to be derived from adapting the task-to-resource mapping while a workflow is executing. This paper describes the use of utility functions to express the relative merits of alternative mappings; in essence, a utility function can be used to give a score to a candidate mapping, and the exploration of alternative mappings can be cast as an optimization problem. In this approach, changing the utility function allows adaptations to be carried out with a view to meeting different objectives. The contributions of this paper include: (i) a description of how adaptive workflow execution can be expressed as an optimization problem where the objective of the adaptation is to maximize a utility function; (ii) a description of how the approach has been applied to support adaptive workflow execution in execution environments consisting of multiple resources, such as grids or clouds, in which adaptations are coordinated across multiple workflows; and (iii) an experimental evaluation of the approach with utility measures based on response time and profit using the Pegasus workflow system.

Workflows are widely used in applications that require coordinated use of computational resources. Workflow definition languages typically abstract over some aspects of the way in which a workflow is to be executed, such as the level of parallelism to be used or the physical resources to be deployed. As a result, a workflow management system has responsibility for establishing how best to map tasks within a workflow to the available resources. As workflows are typically run over shared resources, and thus face unpredictable and changing resource capabilities, there may be benefit to be derived from adapting the task-to-resource mapping while a workflow is executing. This paper describes the use of utility functions to express the relative merits of alternative mappings; in essence, a utility function can be used to give a score to a candidate mapping, and the exploration of alternative mappings can be cast as an optimization problem. In this approach, changing the utility function allows adaptations to be carried out with a view to meeting different objectives. The contributions of this paper include: (i) a description of how adaptive workflow execution can be expressed as an optimization problem where the objective of the adaptation is to maximize a utility function; (ii) a description of how the approach has been applied to support adaptive workflow execution in execution environments consisting of multiple resources, such as grids or clouds, in which adaptations are coordinated across multiple workflows; and (iii) an experimental evaluation of the approach with utility measures based on response time and profit using the Pegasus workflow system. 1.

In this thesis we consider the problem of providing QoS guarantees to Grid users through advance reservation. Advance reservation mechanisms provide the ability to allocate resources to users based on agreed-upon QoS requirements and increase the predictability of a Grid system, yet incorporating such mechanisms into current Grid environments has proven to be a challenging task due to the resulting resource fragmentation. In view of these observations we have devised efficient scheduling algorithms that support advance reservations. We can organize this thesis in two parts. We first use concepts from computational geometry and efficient data structures to present a framework for tackling the resource fragmentation, and for formulating a suite of scheduling strategies. We also develop efficient implementations of the scheduling algorithms that scale to large Grids. We conduct a comprehensive performance evaluation study using simulation, and we present numerical results to demonstrate that our algorithms perform well across several metrics that reflect both user- and system-specific goals. Advance reservations has also been proposed as one mechanisms to provide Grid resource managers with the ability to co-allocate resources. Co-allocation of resources is one

Abstract. We propose a mechanism for the co-allocation of multiple resources in Grid environments. By reserving multiple resources in advance, scientific simulations and large-scale data analyses can efficiently be executed with their desired quality-of-service level. Co-allocating multiple Grid resources in advance poses demanding challenges due to the characteristics of Grid environments, which are (1) incomplete status information, (2) dynamic behavior of resources and users, and (3) autonomous resources ’ management systems. Our co-reservation mechanism addresses these challenges by probing the state of the resources and by enhancing a two-phase commit protocol with timeouts. We performed extensive simulations to evaluate communication overhead of the new protocol and the impact of the timeouts ’ length on the scheduling of jobs as well as on the utilization of the Grid resources. 1