Lo V, Mache J, Windisch K. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP) (Lecture Notes in Computer Science, vol. 1459). Springer: Berlin, 1998; 25--46.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....the original CTC workload most jobs use power of two nodes, whereas in the Jann model jobs are spread evenly between each two consecutive powers of two. Previous work has shown that the fraction of jobs that are powers of two is important for performance, as it is easier to pack power of two jobs [16]. However, in our case this seemed not to make a qualitative difference. It was checked by running the simulations on a modified version of the Jann workload in which the sizes of 80 of the jobs were rounded up to the next power of two. Finally, the size of the system is also important. The Jann ....

V. Lo, J. Mache, and K. Windisch, "A comparative study of real workload traces and synthetic workload models for parallel job scheduling ". In Job Scheduling Strategies for Parallel Processing, pp. 25-- 46, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....structures that create different speedup and synchronization characteristics. As for metrics, we present some problems with slowdown and bounded slowdown that have been proposed recently. 1 Introduction Since the performance of a computer system depends on the workload which it is processing [3, 18], we argue that a workload benchmark suite is needed in order to evaluate and compare the many features of job schedulers for parallel supercomputers. But unlike standard benchmarks suites that consist of a set of representative jobs executed in isolation, a workload benchmark specifies the ....

....of jobs. It is useful because many parallel supercomputers provide schedulers for this type of jobs, and because this type of workload model can be derived from accounting logs. Indeed, a number of such models have already been derived and used in the evaluation of schedulers for parallel systems [8, 15, 18]. 0 5 10 15 20 25 percentage of SDSC Paragon 0 0.5 1 job size Fig. 4. Representative distribution of job sizes, showing dominance of small jobs and jobs using a power of two processors. Data from SDSC Paragon. Interestingly, the accounting logs from many diverse systems show ....

V. Lo, J. Mache, and K. Windisch, "A comparative study of real workload traces and synthetic workload models for parallel job scheduling ". In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 25-- 47, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....but from various correlations between them. Research on these issues is facilitated by the availability of data and models in the Parallel Workloads Archive [60] In addition, there are several documented cases of how workload parameters in uence the outcomes of performance evaluation studies [53, 57, 25]. 2 Data Sources The suggestion that workload modeling should be based on measurements is not new [32, 4] However, for a long time relatively few models based on actual measurements were published. As a result, many performance studies did not use experimental workload models at all (and don t ....

.... larger jobs are also shorter, so packing them rst is statistically similar to using SJF (shortest job rst) 47] But if size and runtime are correlated, and large jobs run longer, scheduling them rst may cause signi cant delays for subsequent smaller jobs, leading to dismal average performance [53]. System Correlation CTC SP2 0:029 KTH SP2 0.011 SDSC SP2 0.145 LANL CM 5 0.211 SDSC Paragon 0.305 Table 2. Correlation coecient of runtime and size for di erent parallel supercomputer workloads. Establishing whether or not a correlation exists is not always easy. The commonly used ....

[Article contains additional citation context not shown here]

V. Lo, J. Mache, and K. Windisch, \A comparative study of real workload traces and synthetic workload models for parallel job scheduling". In Job Scheduling 27 Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 25{ 46, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....the original CTC workload most jobs use power of two nodes, whereas in the Jann model jobs are spread evenly between each two consecutive powers of two. Previous work has shown that the fraction of jobs that are powers of two is important for performance, as it is easier to pack power of two jobs [9]. However, in our case this seemed not to make a qualitative difference (it was checked by running the simulations on a modified version of the Jann workload in which the sizes of 80 of the jobs were rounded up to the next power of two) On the other hand, seemingly unimportant features turned ....

V. Lo, J. Mache, and K. Windisch, "A comparative study of real workload traces and synthetic workload models for parallel job scheduling ". In Job Scheduling Strategies for Parallel Processing, pp. 25-- 46, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....thus complete sooner. Therefore s is the request whose first fit in the allocation list produces the shortest completion time. Consequently, r and s are the same request. 3. Workload Models The performance of a scheduling solution is influenced by the workload submitted to the system [2] 44] [67] [90] Realistic workloads are crucial to establish how scheduling solutions perform in practice. Therefore, in order to evaluate how SA is going to perform in practice, we need to determine the mix of moldable jobs that is likely to compose a supercomputer workload in real life. Workload logs ....

....of how partition sizes are distributed. As Table 6 and Figure 15 show, the distribution of partition sizes seems to be dominated by power of 2 values. This is a relevant characteristic because the fraction of power of 2 jobs in the workload strongly influences the performance of the system [67]. In general, the greater the fraction of powerof 2 jobs in the workload, the better the performance of many scheduling solutions [67] We independently confirmed this result. Workload Power of 2 Jobs ANL 69.87 KTH 73.45 Table 6 Percentage of jobs with a power of 2 partition size 28 ....

[Article contains additional citation context not shown here]

V. Lo, J. Mache, and K. Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In Job Scheduling Strategies for Parallel Processing, Dror Feitelson and Larry Rudolph (eds.), pp. 25-46, Springer Verlag, Lect. Notes Comput. Sci. vol. 1459, 1998. 146

....may create another available slot. Such a slot is backfilled in the same way. The process stops only when no more backfilling can be done. 2.2. Supercomputer Workloads It is well known that the performance of a scheduling solution is influenced by the workload submitted to the system [2] 15] [21] [33] Realistic workloads are thus crucial to establish how scheduling solutions perform in practice. Therefore, in order to evaluate how SA is going to perform in practice, we need to determine the mix of moldable jobs that is likely to compose a supercomputer workload in real life. Workload ....

....of task migration, and (iii) the need to simultaneously make available all processors allocated to a job. In current practice, supercomputer schedulers accept rigid requests [17] 20] 22] 27] and thus much of the research available in the literature assume jobs to be rigid, e.g. 1] 2] 14] [21] [33] Closer to our own work, there has been studies on processor allocation [3] 8] 10] 12] 16] 19] 23] 25] 28] 29] 30] 31] Processor allocation consists of enabling the supercomputer scheduler to select how many processors to allocate to a parallel job based on information about the ....

V. Lo, J. Mache, and K. Windisch. "A comparative study of real workload traces and synthetic workload models for parallel job scheduling". Job Scheduling Strategies for Parallel Processing, Springer-Verlag LNCS vol. 1459, pg. 25-46, 1998.

.... 1.2 16.1 14.2 12.0 feb 0.461 3842 3358 3406 1.4 9.6 8.5 10.9 mar 0.715 4111 5763 5758 0.1 20.1 18.9 6.3 apr 0.788 4300 5616 6184 10.1 28.9 36.0 24.7 Table 1: b)Simulation results for the non SP2 trace les. workloads a ect the quantitative results, but not the qualitative results [18]) The problem is that these workloads are rather complex, and it is not clear exactly what features are the decisive ones. We therefore turn to Talby et al. 26] who made a detailed statistical comparison of workloads and models. That work indicates that the CTC and KTH traces and the Jann model ....

V. Lo, J. Mache, and K. Windisch, \A comparative study of real workload traces and synthetic workload models for parallel job scheduling". In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 25-46, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....the original CTC workload most jobs use power of two nodes, whereas in the Jann model jobs are spread evenly between each two consecutive powers of two. Previous work has shown that the fraction of jobs that are powers of two is important for performance, as it is easier to pack power of two jobs [7]. However, in our case this seemed not to make a qualitative difference. It was checked by running the simulations on a modified version of the Jann workload in which the sizes of 80 of the jobs were rounded up to the next power of two. Finally, the size of the system is also important. The Jann ....

V. Lo, J. Mache, and K. Windisch, "A comparative study of real workload traces and synthetic workload models for parallel job scheduling ". In Job Scheduling Strategies for Parallel Processing, pp. 25--46, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....but more of which jobs are backfilled. We are therefore left with a unique situation in which the workloads dictate the results (the only previous study to systematically check the influence of the workload concluded that workloads affect the quantitative results, but not the qualitative results [18]) The problem is that these workloads are rather complex and it is not clear exactly what features are the decisive ones. We therefore turn to Talby et al. 26] who made a detailed statistical comparison of workloads and models. That work indicates that the CTC and KTH traces and the Jann model ....

V. Lo, J. Mache, and K. Windisch, "A Comparative Study of Real Workload Traces and Synthetic Workload Models for Parallel Job Scheduling," Job Scheduling Strategies for Parallel Processing, D.G. Feitelson and L. Rudolph, eds., pp. 25-46, SpringerVerlag, 1998.

....the use power of two processors. In this case a reasonable scheduling algorithm is to cycle through the di erent sizes, because the jobs of each size pack well together [16] This works well for negatively correlated and even uncorrelated workloads, but is bad for positively correlated workloads [16, 17]. The reason is that under a positive correlation the largest jobs dominate the machine for a long time, blocking out all others. As a result, the average response time of all other jobs grows considerably. But which model actually re ects reality Again, evaluation results depend on the selected ....

V. Lo, J. Mache, and K. Windisch, \A comparative study of real workload traces and synthetic workload models for parallel job scheduling". In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 25{ 46, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....the original CTC workload most jobs use power of two nodes, whereas in the Jann model jobs are spread evenly between each two consecutive powers of two. Previous work has shown that the fraction of jobs that are powers of two is important for performance, as it is easier to pack power of two jobs [6]. However, in our case this seemed not to make a qualitative di erence. It was checked by running the simulations on a modi ed version of the Jann workload in which the sizes of 80 of the jobs were rounded up to the next power of two. Finally, the size of the system is also important. The Jann ....

V. Lo, J. Mache, and K. Windisch, \A comparative study of real workload traces and synthetic workload models for parallel job scheduling". In Job Scheduling Strategies for Parallel Processing, pp. 25-46, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....to that of a large job(J3) There have been many works to investigate this issue as well as to characterize the parallel workload. 13] performed a simulation study with linear as well as uncorrelated relationship. 14] claimed the execution time and the parallelism are linearly related where [15] surveyed several real traces and concluded that there is a positive linear relationship but it is varied and small. Figure 2: Run time snapshot in a 6 cube hypercube but there is also a possibility that one of the other subcubes is allocated to the waiting job. We consider both cases for ....

V.Lo, J.Mache, K.Windisch, "A Comparative Study of Real Workload Traces and Synthetic Workload Models for Parallel Job Scheduling", Job Scheduling Strategies for Parallel Processing, Lecture Notes in Computer Science, Vol.1459, pp.2546, 1998.

....compared with one from the San Diego Supercomputer Center (SDSC) by Windisch et al. WLM 96a] They conclude that the inter arrival time is the most regular of the workload parameters studied, and could be modeled by simpler probabilistic functions . Based on comments by the same authors in [LMW98] one would assume that this means an exponential distribution. The distributions at both machines were quite similar. Due to some limitations in the trace data, the actual data studied were job start times, rather than job arrival times. For interactive jobs, these times are the same. However, ....

....have been reported [CMV94, FN95, WLM 96a, JPF 97] A more interesting and less well understood question is whether job runtimes are correlated with job sizes. A weak positive correlation (i.e. larger jobs run longer) has been reported by several authors [FN95, WLM 96a, Hot96, Fei96, LMW98] but few researchers have modeled this. One slight exception is a study of data from the Cornell Theory Center (CTC) where Hotovy [Hot96] reports a two phase distribution of job duration. Jobs using from 2 16 processors had decreasing runtimes associated with more processors, but jobs using ....

[Article contains additional citation context not shown here]

V. Lo, J. Mache, and K. Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In Dror G. Feitelson and Larry Rudolph, editors, Job Scheduling Strategies for Parallel Processing. Springer-Verlag, 1998. Lect. Notes Comput. Sci. vol. .

....see a strong use of powers of two even in logs that come from machines that do not have queues con gured according to powers of two, e.g. machines running the EASY or Maui schedulers. Finally, we note that power of two jobs have been shown to have a strong impact on performance evaluation results [17], so departing from empirical observations should be justi ed carefully; retaining the observed characteristics is safer. 4.2 The Modeling Procedure Apart from power of two jobs, we also note the high fraction of serial jobs that is typically observed. This motivates us to partition the jobs ....

....CDF is below that of smaller jobs. This is clearly seen in the SDSC logs. In the KTH and LANL logs there is some mixture among the small jobs (in the rst two groups) but not with the large jobs in the third group. 14 0 5 10 15 0 0.2 0.4 0.6 0. 8 1 SDSC95 log runtime [1 , 4] 5 , 16] [17 , 400] 0 5 10 15 0 0.2 0.4 0.6 0.8 1 SDSC96 log runtime [1 , 4] 5 , 16] 17 , 400] 0 5 10 15 0 0.2 0.4 0.6 0.8 1 KTH log runtime [1 , 3] 4 , 8] 9 , 400] 0 5 10 15 0 0.2 0.4 0.6 0.8 1 LANL log runtime [32 , 32] 33 , 128] 129,1024] Figure 4: Distributions of ....

[Article contains additional citation context not shown here]

V. Lo, J. Mache, and K. Windisch, \A comparative study of real workload traces and synthetic workload models for parallel job scheduling". In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 25-46, Springer Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

....task migration, and (iii) the need to simultaneously make available all processors allocated to a job. In current practice, supercomputer schedulers accept rigid requests [20] 23] 25] 28] and thus much of the research available in the literature assume jobs to be rigid, e.g. 1] 2] 16] 19] [24] [33] 35] Closer to our own work, there has been studies on processor allocation [3] 5] 12] 14] 18] 29] 30] 31] 32] Processor allocation consists of selecting how many processors to allocate to a parallel job based on information about the characteristics of the job (e.g. sequential ....

....may create another available slot. Such a slot is backfilled in the same way. The process stops only when no more backfilling can be done. 4.2. Supercomputer Workloads It is well known that the performance of a scheduling solution is influenced by the workload submitted to the system [2] 17] [24] [35] Realistic workloads are thus crucial to establish how scheduling solutions perform in practice. Therefore, in order to evaluate how SA is going to perform in practice, we need to determine the mix of moldable jobs that is likely to compose a supercomputer workload in real life. Workload ....

[Article contains additional citation context not shown here]

V. Lo, J. Mache, and K. Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In Job Scheduling Strategies for Parallel Processing, Feitelson and Rudolph (Eds.), Springer-Verlag Lecture Notes in Computer Science, vol. 1459, pg. 25-46, 1998.

....may create another available slot. Such a slot is backfilled in the same way. The process stops only when no more backfilling can be done. 3.2. Supercomputer Workloads It is well known that the performance of a scheduling solution is influenced by the workload submitted to the system [2] 15] [21] [30] Realistic workloads are thus crucial to establish how scheduling solutions perform in practice. Therefore, in order to evaluate how SA is going to perform in practice, we need to determine the mix of moldable jobs that is likely to compose a supercomputer workload in real life. 5 Workload ....

....supercomputer scheduler to pack another power of 2 job into the schedule than to find an appropriate hole for a job with arbritary partition size. This phenomenon is in consonance with studies that investigated the impact that the fraction of power of 2 jobs in the workload has on performance [21]. 4.2.2. Information Available to SA The information made available to SA varies regarding v, the number of requests that is available to SA, and a, the accuracy of such requests. Recall that the accuracy a is defined as a = te tr, and therefore a small value of a implies that the request ....

[Article contains additional citation context not shown here]

V. Lo, J. Mache, and K. Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In Job Scheduling Strategies for Parallel Processing, Dror Feitelson and Larry Rudolph (eds.), pp. 25-46, Springer Verlag, Lect. Notes Comput. Sci. vol. 1459, 1998.

....a parallel job. This information can for instance be generated by a parallelizing compiler with the aim of statically schedule tasks in order to minimize the overall execution time. Probabilistic Model Two basic approaches are usually used when modeling workloads using the probabilistic model [14]: 1. Use of real workloads traces gathered from scientific production runs on real supercomputers and carefully reconstructed for use in simulation testing. 2. Use of flexible synthetic workload models that use probability distributions to generate workload data. The focus when using this ....

Virginia Lo, Jens Mache, and Kurt Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In Proceedings of the 4rd Workshop on Job Scheduling Strategies for Parallel Processing, volume 1459 of Lecture Notes in Computer Science, pages 25--46, 1998.

No context found.

Lo V, Mache J, Windisch K. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP) (Lecture Notes in Computer Science, vol. 1459). Springer: Berlin, 1998; 25--46.

No context found.

V. Lo, J. Mache, and K. Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In D. G. Feitelson and L. Rudolph, editors, Job Scheduling Strategies for Parallel Processing, pages 25--46. Springer, 1998. Lect. Notes Comput. Sci. vol. 1459.

No context found.

V. Lo, J. Mache, and K. Windisch, "A comparative study of real workload traces and synthetic workload models for parallel job scheduling," in Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph, Eds. Springer Verlag, 1998, vol. 1459, pp. 25--46, Lecture Notes in Computer Science.

No context found.

V. Lo, J. Mache and K. Windisch, A comparative study of real workload traces and synthetic workload models for parallel job scheduling, In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (Eds.), Lecture Notes Computer Science, Vol. 1459, Springer-Verlag, 1998, pp.25-46.

No context found.

V. Lo, J. Mache, and K. Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In Dror G. Feitelson and Larry Rudolph, editors, Job Scheduling Strategies for Parallel Processing, pages 25--46. Springer-Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

No context found.

V. Lo, J. Mache, and K. Windisch, "A comparative study of real workload traces and synthetic workload models for parallel job scheduling," in pp. 25--46, Lecture Notes in Computer Science, Springer Verlag, 1998.

No context found.

V. Lo, J. Mache, and K. Windisch, \A comparative study of real workload traces and synthetic workload models for parallel job scheduling."

No context found.

V. Lo, J. Mache, and K. Windisch. A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In Dror G. Feitelson and Larry Rudolph, editors, Job Scheduling Strategies for Parallel Processing, pages 25--46. Springer-Verlag, 1998. Lect. Notes Comput. Sci. vol. 1459.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC