Results 1 - 10
of
25
GridBot: Execution of Bags of Tasks in Multiple Grids
"... We present a holistic approach for efficient execution of bags-of-tasks (BOTs) on multiple grids, clusters, and volunteer computing grids virtualized as a single computing platform. The challenge is twofold: to assemble this compound environment and to employ it for execution of a mixture of through ..."
Abstract
-
Cited by 25 (2 self)
- Add to MetaCart
(Show Context)
We present a holistic approach for efficient execution of bags-of-tasks (BOTs) on multiple grids, clusters, and volunteer computing grids virtualized as a single computing platform. The challenge is twofold: to assemble this compound environment and to employ it for execution of a mixture of throughput- and performance-oriented BOTs, with a dozen to millions of tasks each. Our generic mechanism allows per BOT specification of dynamic arbitrary scheduling and replication policies as a function of the system state, BOT execution state, and BOT priority. We implement our mechanism in the GridBot system and demonstrate its capabilities in a production setup. GridBot has executed hundreds of BOTs with over 9 million jobs during the last 3 months alone; these have been invoked on 25,000 hosts, 15,000 from the Superlink@Technion community grid and the rest from the Technion campus grid, local clusters, the Open Science Grid, EGEE, and the UW Madison pool.
Dynamic scheduling for heterogeneous Desktop Grids
- in: Proceedings of the 9th International Conference on Grid Computing
"... Desktop Grids have emerged as an important method-ology to harness the idle cycles of millions of participant desktop PCs over the Internet. However, to effectively uti-lize the resources of a Desktop Grid, it is necessary to use scheduling policies suitable for such systems. A scheduling policy mus ..."
Abstract
-
Cited by 12 (2 self)
- Add to MetaCart
(Show Context)
Desktop Grids have emerged as an important method-ology to harness the idle cycles of millions of participant desktop PCs over the Internet. However, to effectively uti-lize the resources of a Desktop Grid, it is necessary to use scheduling policies suitable for such systems. A scheduling policy must be applicable to large-scale systems involving large numbers of machines. Also, the policy must be fault-aware in the sense that it copes with resource volatility. Further adding to the complexity of scheduling for Desktop Grids is the inherent heterogeneity of such systems. Sub-optimal performance would result if the scheduling policy does not take into account information on heterogeneity. In this paper, we suggest and develop several scheduling poli-cies for Desktop Grid systems involving different levels of heterogeneity. In particular, we propose a policy which uti-lizes the solution to a linear programming problem which maximizes system capacity. We consider parallel applica-tions that consist of independent tasks. 1.
Scheduling Algorithms for Multiple Bag-of-Task Applications on Desktop Grids: a Knowledge-Free Approach
"... Desktop Grids are being increasingly used as the execution platform for a variety of applications that can be structured as Bag-of-Tasks (BoT). Scheduling BoT applications on Desktop Grids has thus attracted the attention of the scientific community, and various schedulers tailored towards them have ..."
Abstract
-
Cited by 11 (1 self)
- Add to MetaCart
(Show Context)
Desktop Grids are being increasingly used as the execution platform for a variety of applications that can be structured as Bag-of-Tasks (BoT). Scheduling BoT applications on Desktop Grids has thus attracted the attention of the scientific community, and various schedulers tailored towards them have been proposed in the literature. However, previous work has focused on scheduling a single BoT application at a time, thus neglecting other scenarios in which several users submit multiple BoT applications at the same time. This paper aims at filling this gap by proposing a set of scheduling algorithms able to deal with multiple BoT applications. The performance of these algorithm has been evaluated, by means of simulation, for a large set of operational scenarios obtained by varying both the workload submitted to the Desktop Grid and the characteristics of the involved resources. Our results show that, although there is no a clear winner among the proposed solutions, knowledge-free strategies (that is, strategies that do not require any information concerning the applications or the resources) can provide good performance. 1.
Static Strategies for Worksharing with Unrecoverable Interruptions
"... One has a large workload that is “divisible”—its constituent work’s granularity can be adjusted arbitrarily—and one has access to p remote computers that can assist in computing the workload. The problem is that the remote computers are subject to interruptions of known likelihood that kill all work ..."
Abstract
-
Cited by 6 (2 self)
- Add to MetaCart
(Show Context)
One has a large workload that is “divisible”—its constituent work’s granularity can be adjusted arbitrarily—and one has access to p remote computers that can assist in computing the workload. The problem is that the remote computers are subject to interruptions of known likelihood that kill all work in progress. One wishes to orchestrate sharing the workload with the remote computers in a way that maximizes the expected amount of work completed. Strategies for achieving this goal, by balancing the desire to checkpoint often, in order to decrease the amount of vulnerable work at any point, vs. the desire to avoid the context-switching required to checkpoint, are studied. Strategies are devised that provably maximize the expected amount of work when there is only one remote computer (the case p = 1). Results suggest the intractability of such maximization for higher values of p, which motivates the development of heuristic approaches. Heuristics are developed that replicate works on several remote computers, in the hope of thereby decreasing the impact of work-killing interruptions. The quality of these heuristics is assessed through exhaustive simulations.
SpeQuloS: A QoS Service for BoT Applications Using Best Effort Distributed Computing Infrastructures
, 2012
"... Exploitation of Best E ort Distributed Computing Infrastructures (BE-DCIs) allow operators to maximize the utilization of the infrastructures, and users to access the unused resources at relatively low cost. Because providers do not guarantee that the computing resources remain available to the user ..."
Abstract
-
Cited by 5 (1 self)
- Add to MetaCart
Exploitation of Best E ort Distributed Computing Infrastructures (BE-DCIs) allow operators to maximize the utilization of the infrastructures, and users to access the unused resources at relatively low cost. Because providers do not guarantee that the computing resources remain available to the user during the entire execution of their applications, they o er a diminished Quality of Service (QoS) compared to traditional infrastructures. Pro ling the execution of Bag-of-Tasks (BoT) applications on several kinds of BE-DCIs demonstrates that their task completion rate drops near the end of the execution. In this report, we present the SpeQuloS service which enhances the QoS of BoT applications executed on BE-DCIs by reducing the execution time, improving its stability, and reporting to users a predicted completion time. SpeQuloS monitors the execution of the BoT on the BE-DCIs, and dynamically supplies fast and reliable Cloud resources when the critical part of the BoT is executed. We present the design and development of the framework and several strategies to decide when and how Cloud resources should be provisioned. Performance evaluation using simulations shows that SpeQuloS ful ll its objectives. It speeds-up the execution of BoTs, in the best cases by a factor greater than 2, while offloading less than 2.5 % of the workload to the Cloud. We report on preliminary
A Fair Decentralized Scheduler for Bag-of-tasks Applications on Desktop Grids
, 2010
"... Desktop Grids have become very popular nowadays, with projects that include hundred of thousands computers. Desktop grid scheduling faces two challenges. First, the platform is volatile, since users may reclaim their computer at any time, which makes centralized schedulers inappropriate. Second, des ..."
Abstract
-
Cited by 5 (0 self)
- Add to MetaCart
(Show Context)
Desktop Grids have become very popular nowadays, with projects that include hundred of thousands computers. Desktop grid scheduling faces two challenges. First, the platform is volatile, since users may reclaim their computer at any time, which makes centralized schedulers inappropriate. Second, desktop grids are likely to be shared among several users, thus we must be particularly careful to ensure a fair sharing of the resources. In this paper, we propose a decentralized scheduler for bag-of-tasks applications on desktop grids, which ensures a fair and efficient use of the resources. It aims to provide a similar share of the platform to every application by minimizing their maximum stretch, using completely decentralized algorithms and protocols. After presenting our algorithms, we evaluate them through extensive simulations. We compare our solution to already existing centralized ones under similar conditions, and show that its performance is close to the best centralized algorithms. 1 1
Power-Aware Linear Programming Based Scheduling for Heterogeneous Computer Clusters
, 2011
"... In the past few years, scheduling for computer clusters has become a hot topic. The main focus has been towards achieving better performance. It is true that this goal has been attained to a certain extent, but on the other hand, it has been at the expense of increased energy consumption and con-seq ..."
Abstract
-
Cited by 5 (0 self)
- Add to MetaCart
(Show Context)
In the past few years, scheduling for computer clusters has become a hot topic. The main focus has been towards achieving better performance. It is true that this goal has been attained to a certain extent, but on the other hand, it has been at the expense of increased energy consumption and con-sequent economic and environmental costs. As these clusters are becoming more popular and complex, reducing energy consumption in such systems has become a necessity. Several power-aware scheduling policies have been proposed for homogeneous clusters. In this work, we propose a new policy for heterogeneous clusters. Our simulation experiments show that using our proposed policy results in significant reduction in energy consumption while performing very competitively in heterogeneous clusters. 1.
Decentralized Load Balancing for Heterogeneous Grids.
, 2009
"... Abstract-Several decentralized load balancing policies have been proposed to address the issue of scalability in grids. However, the communication overhead incurred in exchanging state information remains a burden. In this paper, we propose a dynamic, decentralized load balancing policy which perfo ..."
Abstract
-
Cited by 4 (0 self)
- Add to MetaCart
(Show Context)
Abstract-Several decentralized load balancing policies have been proposed to address the issue of scalability in grids. However, the communication overhead incurred in exchanging state information remains a burden. In this paper, we propose a dynamic, decentralized load balancing policy which performs very competitively in heterogeneous grids. The policy uses an effective mechanism for state information exchange, which significantly reduces the communication overhead, while quickly updating the state information in a decentralized fashion.
Scheduling Parallel Iterative Applications on Volatile Resources
"... Abstract—In this paper we study the execution of iterative applications on volatile processors such as those found on desktop grids. We develop master-worker scheduling schemes that attempt to achieve good trade-offs between worker speed and worker availability. A key feature of our approach is that ..."
Abstract
-
Cited by 3 (2 self)
- Add to MetaCart
(Show Context)
Abstract—In this paper we study the execution of iterative applications on volatile processors such as those found on desktop grids. We develop master-worker scheduling schemes that attempt to achieve good trade-offs between worker speed and worker availability. A key feature of our approach is that we consider a communication model where the bandwidth capacity of the master for sending application data to workers is limited. This limitation makes the scheduling problem more difficult both in a theoretical sense and in a practical sense. Furthermore, we consider that a processor can be in one of three states: available, down, or temporarily preempted by its owner. This preempted state also complicates the scheduling problem. In practical settings, e.g., desktop grids, master bandwidth is limited and processors are temporarily reclaimed. Consequently, addressing the aforementioned difficulties is necessary for successfully deploying masterworker applications on volatile platforms. Our first contribution is to determine the complexity of the scheduling problem in its off-line version, i.e., when processor availability behaviors are known in advance. Even with this knowledge, the problem is NP-hard, and cannot be approximated within a factor 8/7. Our second contribution is a closed-form formula for the expectation of the time needed by a worker to complete a set of tasks. This formula relies on a Markovian assumption for the temporal availability of processors, and is at the heart of some heuristics that aim at favoring “reliable” processors in a sensible manner. Our third contribution is a set of heuristics, which we evaluate in simulation. Our results provide guidance to selecting the best strategy as a function of processor state availability versus average task duration. I.
MGST: a framework for the performance evaluation of Desktop Grids
- in: Proceedings of the 24th International Parallel and Distributed Processing Symposium
, 2009
"... Desktop Grids are rapidly gaining popularity as a cost-effective computing platform for the execution of applications with extensive computing needs. As opposed to grids and clus-ters, these systems are characterized by having a non-dedicated infrastructure. These unique characteristics need to be c ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
(Show Context)
Desktop Grids are rapidly gaining popularity as a cost-effective computing platform for the execution of applications with extensive computing needs. As opposed to grids and clus-ters, these systems are characterized by having a non-dedicated infrastructure. These unique characteristics need to be consid-ered in developing resource management strategies for Desktop Grids. Several frameworks for the performance evaluation of resource management strategies have been suggested for grids. However, similar projects for Desktop Grids are still lacking. This paper presents MGST, the first performance testing frame-work for Desktop Grids. We discuss the design of the tool and show how it can be used to analyze and improve the perfor-mance of an existing Desktop Grid scheduling policy. 1.