To provide Quality of Service (QoS) guarantee in distributed services, it is necessary to reserve multiple computing and communication resources for each service session. Meanwhile, techniques have been available for the reservation and enforcement of various types of resources. Therefore, there is a need to create an integrated framework for coordinated multi-resource reservation. One challenge in creating such a framework is the complex relation between the end-to-end application-level QoS and the corresponding end-to-end resource requirement. Furthermore, the goals of (1) providing the best end-to-end QoS for each distributed service session and (2) increasing the overall reservation success rate of all service sessions are in conflict with each other. In this paper, we present a QoS and contention-aware framework of end-to-end multi-resource reservation for distributed services. The framework assumes a reservation enabled environment, where each type of resource can be reserved. The framework consists of (1) a component-based QoS-Resource Model, (2) a runtime system architecture for coordinated reservation, and (3) a runtime algorithm for the computation of end-to-end multi-resource reservation plans. The algorithm provides a solution to alleviating the conflict between the QoS of an individual service session and the success rate of all service sessions. More specifically, for each service session, the algorithm computes an end-to-end reservation plan, such that it guarantees the highest possible end-to-end QoS level under the current end-to-end resource availability, and requires the lowest percentage of bottleneck resource(s) among all feasible reservation plans. Our simulation results show excellent performance of this algorithm.

Abstract—In scheduling a large number of user jobs for parallel execution on an open-resource Grid system, the jobs are subject to system failures or delays caused by infected hardware, software vulnerability, and distrusted security policy. This paper models the risk and insecure conditions in Grid job scheduling. Three risk-resilient strategies, preemptive, replication, and delay-tolerant, are developed to provide security assurance. We propose six risk-resilient scheduling algorithms to assure secure Grid job execution under different risky conditions. We report the simulated Grid performances of these new Grid job scheduling algorithms under the NAS and PSA workloads. The relative performance is measured by the total job makespan, Grid resource utilization, job failure rate, slowdown ratio, replication overhead, etc. In addition to extending from known scheduling heuristics, we developed a new space-time genetic algorithm (STGA) based on faster searching and protected chromosome formation. Our simulation results suggest that, in a wide-area Grid environment, it is more resilient for the global job scheduler to tolerate some job delays instead of resorting to preemption or replication or taking a risk on unreliable resources allocated. We find that delay-tolerant Min-Min and STGA job scheduling have 13-23 percent higher performance than using risky or preemptive or replicated algorithms. The resource overheads for replicated job scheduling are kept at a low 15 percent. The delayed job execution is optimized with a delay factor, which is 20 percent of the total makespan. A Kiviat graph is proposed for demonstrating the quality of Grid computing services. These riskresilient job scheduling schemes can upgrade Grid performance significantly at only a moderate increase in extra resources or scheduling delays in a risky Grid computing environment. Index Terms—Grid computing, job scheduling heuristics, genetic algorithm, replication scheduling, risk resilience, NAS and PSA benchmarks, performance metrics, distributed supercomputing. 1

Abstract. How to build the mutual trust among Grid resources sites is crucial to secure distributed Grid applications. We suggest enhancing the trust index of resource sites by upgrading their intrusion defense capabilities and checking the success rate of jobs running on the platforms. We propose a new fuzzy-logic trust model for securing Grid resources. Grid security is enforced through trust update, propagation, and integration across sites. Fuzzy trust integration reduces platform vulnerability and guides the defense deployment across Grid sites. We developed a SeGO scheduler for trusted Grid resource allocation. The SeGO scheduler optimizes the aggregate computing power with security assurance under fixed budget constraints. The effectiveness of the scheme was verified by simulation experiments. Our results show up to 90% enhancement in site security. Compared with no trust integration, our scheme leads to 114 % improvement in Grid performance/cost ratio. The job drop rate reduces by 75%. The utilization of Grid resources increased to 92.6 % as more jobs are submitted. These results demonstrate significant performance gains through optimized resource allocation and aggressive security reinforcement. 1.

Air Quality Forecasting (AQF) is a new discipline that attempts to reliably predict atmospheric pollution. An AQF application has complex workflows and in order to produce timely and reliable forecast results, each execution requires access to diverse and distributed computational and storage resources. Deploying AQF on grids is one option to satisfy such needs, but requires the related grid middleware to support automated workflow scheduling and execution on grid resources. In this paper, we analyze the challenges in deploying an AQF application in a campus grid environment and present our current efforts to develop a general solution for grid-enabling scientific workflow applications in the GRACCE project. In GRACCE, an application’s workflow is described using GAMDL, a powerful dataflow language for describing application logic. The GRACCE metascheduling architecture provides the functionalities required for co-allocating grid resources for workflow tasks, scheduling the workflows and monitoring their execution. By providing an integrated framework for modeling and metascheduling scientific workflow applications on grid resources, we make it easy to build a customized environment with end-to-end support for application grid deployment, from the management of an application and its dataset, to the automatic execution and analysis of its results.

Air Quality Forecasting (AQF) is a new discipline that attempts to reliably predict atmospheric pollution. The application has complex workflow and in order to produce timely and reliable forecast results daily, each execution requires multiple computational and storage resources to be simultaneously and collaboratively available. Deploying AQF on grid is one option to satisfy such needs, but requires related grid middleware to support automated application-specific scheduling and execution on grid resources. This paper presents our initial experience of deploying AQF on a campus grid environment and our current efforts of developing a solution of grid-enabling AQF-like applications in Gracce project. Gracce has the goal to provide domain users a grid platform supporting from the management of an application and its dataset, to the automatic execution and viewing of results. In Gracce, application workflow is described using GAMDL, a powerful data-flow language for domain users in describing application logics. The Gracce metascheduler architecture, which includes a workflow-orchestrated metascheduler, an event-driven workflow engine, and an execution runtime system provides the required functionalities of scheduling application workflow in global level and coordinating workflow executions. 1

The ultimate goal of grid technologies is to materialize the vision of grids as virtual supercomputers of unprecedented power, through utilization of geographically disperse distributively owned resources. Despite the overwhelming success of grids in running pleasantly parallel tasks, there still exists a large set of demanding applications considered the exclusive prerogative of real supercomputers. A few examples include complex systems and weather simulations, computational fluid dynamics and other tightly coupled parallel applications. These rely on a static execution environment with predictable performance, provided through efficient co-allocation of a large number of reliable homogeneously interconnected resources. In this paper, we describe a novel quasiopportunistic supercomputer system that enables execution of demanding parallel applications in grids through identification and implementation of the set of key technologies required to bridge the gap between grids and supercomputers. These technologies include an economic incentive-based framework for establishing and maintaining grid-wise allocation agreements; a co-allocation subsystem that is integrated with the economic framework and enhanced by communication topology-aware allocation mechanisms; a fault tolerant message passing library that hides the failures of the underlying resources; and data pre-staging orchestration.

Service providers of the future could dynamically negotiate for, and create their infrastructure on Grid based utility computing and communication providers. Such commercialisation of large scale gridsystems requires the provision of mechanisms to share the wide pool of Grid brokered resources such as computers, software, licences and peripherals amongst many users and organisations. Quickly and efficiently servicing resource requests is critical to the efficiency of such Grid based utility computing and communication providers. However, distributed resource negotiation is itself a contributor to lower system utilisation, as the negotiation process introduces latency and reservation uncertainty in the system. The CORA architecture is a market based resource negotiation system that utilises a Vickrey auction to make allocations of resource requests to resource providers. The architecture utilises a novel combination techniques to improve utilisation, including oversubscription, coallocation, just-in-time reallocation and a novel flexible contract structure. This paper introduces two significant improvements to the CORA architecture. Firstly, redundant contracts are generated to resolve the problem of post bid unavailability of bidders. Secondly, this paper utilises a new auction architecture that does not require the auctioneer to be trusted. The advantage is that any entity (untrusted or otherwise) can conduct a verifiable and privacy preserving Vickrey auction, removing the need for a trusted and privileged auctioneer within the system.

The security of Grid sites can be enhanced by upgrading its intrusion defense capabilities against its previous job success rate on Grid platforms. A new fuzzylogic trust model is proposed for distributed security enforcement across multiple Grid resources sites. The design is aimed at securing Grid resources with optimized resources subject to budget constraints. The performance of trusted Grid computing is verified by simulated trust integration over multiple Grid resource sites. The SARAH scheme scales well with increasing number of divisible user jobs and can sustain high efficiency, as more resource sites are added. Integrated trust and resource optimization make it possible to accommodate all user applications with low job drops and short waiting time. As a result, the Grid resources are better utilized for distributed execution of large number of user jobs.

Abstract: A new trust-based model is developed for optimizing resource allocation in a distributed Grid computing environment. Highly shared resources in a Grid create the insecurity and dependability concerns that hinder distributed applications. We model the Grid resource allocation process as a multi-objective integer-programming problem. Trusted Grid computing power is maximized over multiple resources. At the same time, the aggregate cost of Grid services is minimized to yield a high performance/cost ratio. Our new Security-Assured Resource Allocation (SARA) scheme enables dynamic resources under the security constraints. The Grid resources are assured with a distributed security infrastructure. First, we present the mathematical model for SARA optimization process. Then we use the Simplex method and branch-and-bound method to obtain optimal or suboptimal solutions. We illustrate the ideas with a working example of a pool of 11 host machines from 2 resource sites. For k resource sites with N machine hosts, our SARA algorithm has a search complexity O(2 k-1 N/k), which is much lower than O(2 N) in exhaustive search for the optimal solutions. Our model can be applied to secure many Grid applications in scientific supercomputing or in cyberspace information services. Index Terms: Computational grids, distributed supercomputing, resource allocation, network security, trust models, dependable computing, integer programming, Simplex method, and branch-and-bound method. 1.

Abstract—In hosting environments such as IaaS clouds, desirable application performance is usually guaranteed through the use of Service Level Agreements (SLAs), which specify minimal fractions of resource capacities that must be allocated for unencumbered use for proper operation. Arbitrary colocation of applications with different SLAs on a single host may result in inefficient utilization of the host’s resources. In this paper, we propose that periodic resource allocation and consumption models – often used to characterize real-time workloads – be used for a more granular expression of SLAs. Our proposed SLA model has the salient feature that it exposes flexibilities that enable the infrastructure provider to safely transform SLAs from one form to another for the purpose of achieving more efficient colocation. Towards that goal, we present MORPHOSYS: a framework for a service that allows the manipulation of SLAs to enable efficient colocation of arbitrary workloads in a dynamic setting. We present results from extensive trace-driven simulations of colocated Video-on-Demand servers in a cloud setting. These results show that potentially-significant reduction in wasted resources (by as much as 60%) are possible using MORPHOSYS. I.