Talby D, Feitelson DG. Supporting priorities and improving utilization of the IBM SP2 scheduler using slack-based backfilling. Proceedings of the IPPS. IEEE Computer Society Press: Los Alamitos, CA, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....it reduces resource fragmentation This work was partially supported by the National Science Foundation under grants EIA 9977030, EIA 9974992, CCR 0098278, and ACI 0090221. and increases system utilization, back lling has been proposed as a more e cient alternative to simple FCFS schedulers [7, 10]. Users are expected to provide nearly accurate estimates of job execution times. Using these estimates, the scheduler rearranges the waiting queue, allowing short jobs to move ahead of long jobs provided certain previously submitted jobs are not delayed. Various versions of back lling have been ....

Talby, D. and Feitelson, D.G.: Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler Using Slack-Based Back lling. Proceedings of the 13th International Parallel Processing Symposium. (1999) 513-517.

....is more dominant. Since the two goals of reduced run time and reduced wait time are conflicting, our challenge is to design a strategy that achieves a reduction in overall turn around time. There has been considerable prior research into each of the two topics of i) scheduling of parallel jobs [1,2,6,10,14,19,20,23,24,26], and ii) contiguous node allocation strategies [3,4,5,11,13,27] There have also been a few studies that have considered both these issues in combination [12,15,18] However, only [15] addresses the impact of contiguous node allocation schemes in conjunction with a job scheduling policy that ....

....axis. Each job can be thought of as a rectangle whose length is the user estimated run time and width is the number of processors required. The simplest way to schedule jobs fairly is to use the First ComeFirst Served (FCFS) policy. This approach suffers from low system utilization. Backfilling [14,19,24] was proposed to improve the system utilization and has been implemented is several production schedulers [9] Backfilling works by identifying holes in the 2D chart and moving forward smaller jobs that fit those holes. There are two common variations to backfilling conservative and ....

D. Talby and D. G. Feitelson, Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler Using Slack-Based Backfilling. 13th Intl. Parallel Processing Symp., pp. 513-517, Apr 1999

....run time and width is the number of processors required. Parallel job scheduling strategies have been widely studied in the past [13, 14, 15, 7, 1] The simplest way to schedule jobs is to use the First Come First Served (FCFS) policy. This approach suffers from low system utilization. Backfilling [2, 8, 6] was proposed to improve system utilization and has been implemented in several production schedulers [5] Backfilling works by identifying holes in the 2D chart and moving forward smaller jobs that fit those holes. There are two common variations to backfilling conservative and aggressive ....

....FCFS Selective D = 15 Min 15 Min 1 Hr 1Hr 4Hrs 4Hrs 16Hrs 16Hrs of Jobs 4 2 4 1.5 2 1 1.5 Figure 7: Fairness comparison of various schemes. The Selective backfilling schemes are better than or comparable to FCFS EASY with respect to fairness. on standard backfilling include [8, 15]. In [8] an approach is developed where each job is associated with a deadline (based on its priority) and a job is allowed to backfill provided it does not delay any job in the queue by more than that job s slack. Such an approach provides greater flexibility to the scheduler compared to ....

[Article contains additional citation context not shown here]

D. Talby and D. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP Scheduler Using Slack-Based Backfilling. In Proceedings of the 13th International Parallel Processing Symposium, 1999.

....Hence, much work has been done in the field of improving or developing new scheduling algorithms and policies in general. Some examples to mention are: gang scheduling [2] combined with migration and backfilling [20] several backfilling variants (conservative [4] EASY [13] or slack based [18]) or a tool for predicting job runtimes [14] Also, research for explicit machines was done, e.g. for the IBM SP2 and the LoadLeveler System [18, 11] or the IBM ASCI Blue [5, 10] It is common to use simulation environments for evaluating scheduling algorithms. Job sets are often based on ....

.... are: gang scheduling [2] combined with migration and backfilling [20] several backfilling variants (conservative [4] EASY [13] or slack based [18] or a tool for predicting job runtimes [14] Also, research for explicit machines was done, e.g. for the IBM SP2 and the LoadLeveler System [18, 11], or the IBM ASCI Blue [5, 10] It is common to use simulation environments for evaluating scheduling algorithms. Job sets are often based on trace information from real machines. Especially for that purpose a Parallel Workload Archive [17] was established. During the last years such simulation ....

D. Talby and D. G. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler Using Slack-Based Backfilling. TR 98-13, Hebrew University, Jerusalem, April 1999.

....these software systems is the jobscheduler. In a production environment the performance of the scheduler is primarily responsible for a good or bad overall machine performance and user acceptance. Developing and improving job schedulers is often done with the aid of simulations (Feitelson et al. [3, 9, 12] for the IBM SP2, or by Franke, Moreira et al. 4, 8] for the ASCI BluePacific) This paper presents a similar approach: improving the job scheduler of CCS (Computing Center Software) for our Fujitsu Siemens hpcLine Cluster. CCS [6, 7] was developed at the PC over the last years to manage and ....

D. Talby and D. G. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler Using Slack-Based Backfilling. TR 98-13, Hebrew University, Jerusalem, April 1999.

....Longest Job First (LJF) each combined with conservative backfilling. As the scheduler has a strong influence on the system performance, it is worth while to improve the performance of the scheduling strategy. In the past preliminary evaluations based on simulations were done by Feitelson et al. [1, 10, 12] for the IBM SP2, or by Franke, Moreira et al. 4, 9] for the ASCI Blue Pacific. The remainder of this paper is organized as follows. Section 2 describes the scheduling strategies and backfilling in detail. The two job sets for the evaluation are described in Section 3. After the performance ....

D. Talby and D. G. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler Using Slack-Based Backfilling. Tr 98-13, Hebrew University, Jerusalem, April 1999.

....Longest Job First (LJF) each combined with conservative backfilling. As the scheduler has a strong influence on the system performance, it is worth while to improve the performance of the scheduling strategy. In the past preliminary evaluations based on simulations were done by Feitelson et al. [1, 14] for the IBM SP2, or by Franke, Moreira et al. 7, 12] for the ASCI Blue Pacific. The remainder of this paper is organized as follows. Section 2 describes the scheduling strategies and backfilling in detail. The two job sets for the evaluation are described in Section 3. After the performance ....

D. Talby and D. G. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler Using Slack-Based Backfilling. TR 98-13, Hebrew University, Jerusalem, April 1999.

....in several system schedulers [10, 14, 23] and has been shown to be effective in increasing the system utilization [11] Feitelson and Weil [7] show that the performance of FCFS backfill is relatively insensitive to the number of jobs that are given scheduled start times. Talby and Feitelson [25] propose a FCFS backfill policy with a more flexible backfill deadline, and show that this flexible backfill deadline improves average wait by 10 20 for twelve monthly IBM SP 2 workloads. Flexible backfill deadlines are not evaluated in this paper. Leinberger et al. 12] propose a modified ....

D. Talby and D. G. Feitelson. Supporting priorities and improving utilization of the IBM SP2 scheduler using slack-based backfilling. Proc. 13th Int'l. Parallel Processing Symp., San Juan, Apr. 1999, pp. 513-517.

....slowdown, and hence overall average slowdown, more than large jobs. We have presented four techniques for improving schedule quality in backfilling schedulers. The first systematically stretches estimated execution times in order to open up larger gaps for backfilling. Recent work by Talby et al. [8] and Hollingsworth et al. 4] takes similar approaches. The former also accommodates priorities, while the latter targets cooperation between loosely coupled systems. Our second technique explicitly targets slowdown by sorting waiting jobs by length. The resulting schedules are highly efficient ....

David Talby and Dror G. Feitelson. Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backfilling. In Proceedings of the 13th International Parallel Processing Symposium, 1999.

....multiple resource requirements, scheduling dependencies and job priorities. Not surprisingly this problem is NP hard and many heuristic techniques have been developed to approximate a good solution. Examples of recent work for non preemptive, preemptive and multiple resource schemes include [4, 6, 5, 7]. Particular scheduling strategies depend on the availability of certain key system functionality. As shown in Table 1, the systems examined in this work di er considerably in available functionality. Checkpoint restart is the ability to save to disk and subsequently restore a running job. ....

Talby D., Feitelson D. G.: Supporting priorities and improving utilization of the IBM SP2 scheduler using slack-based backlling, 13th International Parallel Processing Symposium (1999) 513

....the efficiency, E is unity. Here the numerator and denominator are in units of CPU seconds. The problem of optimal packing is NP complete, but several effective heuristic techniques have been developed. Examples of recent work in non preemptive, preemptive and multiple resource schemes include [8, 10, 4, 6]. Particular scheduling strategies depend on the availability of certain key system functionalities. As shown in Table 1, the currently available T3E and SP systems at NERSC differ considerably in this regard. Here checkpoint restart is the ability to save to disk and subsequently restore a ....

D. Talby and D. G. Feitelson, "Supporting Priorities and Improving Utilization of the IBM SP2 Scheduler using Slack-Based Backfilling", 13th International Parallel Processing Symposium, IEEE, 1999, pg. 513.

....multiple resource requirements, scheduling dependencies and job priorities. Not surprisingly this problem is NP hard and many heuristic techniques have been developed to approximate a good solution. Examples of recent work for nonpreemptive, preemptive and multiple resource schemes include [4, 5, 6, 7]. Particular scheduling strategies depend on the availability of certain key system functionality. As shown in Table I, the systems examined in this work differ considerably in available functionality. Checkpoint restart is the ability to save to disk and subsequently restore a running job. ....

D. Talby and D. G. Feitelson. Supporting priorities and improving utilization of the IBM SP2 scheduler using slack-based backfilling. In 13th International Parallel Processing Symposium, page 513. IEEE, 1999.

....multiple resource requirements, scheduling dependencies and job priorities. Not surprisingly this problem is NP hard and many heuristic techniques have been developed to approximate a good solution. Examples of recent work for non preemptive, preemptive and multiple resource schemes include [4, 5, 6, 7]. Particular scheduling strategies depend on the availability of certain key system functionality. As shown in Table I, the systems examined in this work differ considerably in available functionality. Checkpoint restart is the ability to save to disk and subsequently restore a running job. ....

D. Talby and D. G. Feitelson. Supporting priorities and improving utilization of the IBM SP2 scheduler using slack-based backfilling. In 13th International Parallel Processing Symposium, page 513. IEEE, 1999.

....any waiting job by filling gaps in the schedule. The work in [11] backs our decision in showing that conservative backfilling is equivalent to aggressive backfilling which only tries not to delay the first waiting job in the queue. A further improvement could be obtained by slack based backfilling [38] which uses so called slack values reflecting the relative importance of the jobs. This techniques needs both user selected and administrative priorities which are currently not available to Winner s scheduler. Another possible improvement could so far not be implemented in Winner s batch ....

D. Talby and D.G. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP Scheduler Using SlackBased Backfilling. In Proc. joint International Parallel Processing Symposium, Symposium on Parallel & Distributed Processing (IPPS/SPDP '99), Apr. 1999, San Juan, Puerto Rico.

No context found.

D. Talby and D. G. Feitelson, "Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backfilling ". In 13th Intl. Parallel Processing Symp., pp. 513--517, Apr 1999.

No context found.

D. Talby and D. G. Feitelson, "Supporting Priorities and Improving Utilization of the IBM SP Scheduler Using Slack-Based Backfilling". In 13th Intl. Parallel Processing Symp. (IPPS), pp 513-517, Apr 1999.

....the two approaches is better. Our case study is a set of simulations designed to answer this question. It should be noted that other variants of backfilling have also been suggested. The main ideas are to allow the scheduler more flexibility, by allowing it some slack in fulfilling reservations [24, 23], or by considering different sets of jobs for backfilling [21] Backfilling has also been integrated with multiple queue scheduling [14] and with gang scheduling [25] But here we focus on the two simpler variants. 3 Evaluation Methodology There are many ways to evaluate the performance of ....

D. Talby and D. G. Feitelson, "Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backfilling ". In 13th Intl. Parallel Processing Symp., pp. 513--517, Apr 1999.

....slowdown exceeds some threshold. If the threshold is chosen judiciously, only the most needy jobs get a reservation. Additional variants of back lling allow the scheduler more exibility. Talby and Feitelson presented slack based back lling, an enhanced back ll scheduler that supports priorities [6]. These priorities are used to assign each waiting job a slack, which determines how long it may have to wait before running: important jobs will have little slack in comparison with others. Back lling is allowed only if the back lled job does not delay any other job by more than that job s slack. ....

D. Talby and D. G. Feitelson, "Supporting Priorities and Improving Utilization of the IBM SP Scheduler Using Slack-Based Backlling". In 13th Intl. Parallel Processing Symp. (IPPS), pp 513-517, Apr 1999.

....but that in fact this has the potential to be bene cial, because back lling works better if it is allowed some exibility. Even a simple approach of just multiplying user estimates by a constant leads to improvements. More sophisticated approaches, such as that recently proposed by Talby et al. [25], may be even better. Acknowledgements This research was supported by the Ministry of Science and Technology and by the Israel Science Foundation founded by the Israel Academy of Sciences and Humanities. The workload log from the CTC SP2 was graciously provided by the Cornell Theory Center, a ....

D. Talby and D. G. Feitelson, \Supporting priorities and improving utilization of the IBM SP scheduler using slack-based back lling". In 13th Intl. Parallel Processing Symp., pp. 513-517, Apr 1999.

....in fact, this has the potential to be beneficial because backfilling works better if it is allowed some flexibility. Even a simple approach of just multiplying user estimates by a constant leads to improvements. More sophisticated approaches, such as that recently proposed by Talby and Feitelson [25], may be even better. ACKNOWLEDGMENTS This research was supported by the Ministry of Science and Technology and by the Israel Science Foundation founded by the Israel Academy of Sciences and Humanities. The workload log from the CTC SP2 was graciously provided by the Cornell Theory Center, a ....

D. Talby and D.G. Feitelson, "Supporting Priorities and Improving Utilization of the IBM SP Scheduler Using Slack-Based Backfilling," Proc. 13th Int'l Parallel Processing Syrup., pp. 513-517, Apr. 1999.

....but that in fact this has the potential to be bene cial, because back lling works better if it is allowed some exibility. Even a simple approach of just multiplying user estimates by a constant leads to improvements. More sophisticated approaches, such as that recently proposed by Talby et al. [25], may be even better. 6 Acknowledgements This research was supported by the Ministry of Science and Technology and by the Israel Science Foundation founded by the Israel Academy of Sciences and Humanities. The workload log from the CTC SP2 was graciously provided by the Cornell Theory Center, a ....

D. Talby and D. G. Feitelson, \Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backlling". In 13th Intl. Parallel Processing Symp., pp. 513-517, Apr 1999.

No context found.

Talby D, Feitelson DG. Supporting priorities and improving utilization of the IBM SP2 scheduler using slack-based backfilling. Proceedings of the IPPS. IEEE Computer Society Press: Los Alamitos, CA, 1999.

No context found.

D. Talby and D. G. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP2 scheduler using Slack Based Backfilling. In the Proceedings of the 13th Intl. Parallel Processing Symposium, pages 513--517, April 1997.

No context found.

D. Talby and D. G. Feitelson. Supporting priorities and improving utilization of the IBM SP2 scheduler using slackbased backfilling. In 13th Intl. Parallel Processing Symp., pages 513--517, April 1999.

No context found.

D. Talby and D. Feitelson. Supporting Priorities and Improving Utilization of the IBM SP Scheduler Using Slack-Based Backfilling. In Proceedings of the 13th International Parallel Processing Symposium, 1999.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC