A. B. Downey and D. G. Feitelson. The Elusive Goal of Workload Characterization. Technical report, Hebrew University, Jerusalem, March 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....72 80 88 96 104 112 120 128 136 144 152 160 168 176 184 192 200 208 216 224 232 240 Fig. 2. Resource consumption of jobs in the CTC syn workload, sorted by requested resources. 4. 4 Fragmentation Parameters The analysis of the used workloads shows a tendency of jobs to size with the power of 2 [3], as seen in Figure 2. Herein the resource consumption is summed up for all jobs of a specific width (i.e. the number of requested resources) for the ctc syn workload. The other workloads show a similar behavior. Because of this power of 2 focus we also used power of 2 values for the lower bound ....

A. B. Downey and D. G. Feitelson. The Elusive Goal of Workload Characterization. Technical report, Hebrew University, Jerusalem, March 1999.

.... results are those that deal with: i) The predictability of supercomputer schedulers [Downey 97c] Feitelson 98] Gibbons 97] Smith 98] Smith 99] ii) The impact the accuracy of the requests has on scheduling [Feitelson 98] Zotkin 99] and iii) Modeling supercomputers workloads [Downey 97b] Downey 99] Feitelson 97b] Jann 97] Predictability in Resource Scheduling Within the operating systems community, there has been considerable effort to make resource scheduling more predictable [Fong 95] Stoica 96] Waldspurger 94] Waldspurger 95] However, many of these efforts primarily aim to ....

....time reaches the moment an application finishes, the simulator informs S that the application is done (and hence that the processors it occupied are now free) Unfortunately, simulating the request stream generated by the users is not that easy. In fact, it is a very hard task [Chapin 99] Downey 99] With this in mind, we ve opted to use the submission log of real supercomputers in our simulations. In particular, we use the following submission logs: San Diego Supercomputer Center (SDSC) SP2, 128 nodes, January to April 1999; Cornell Theory Center (CTC) SP2, 430 nodes, July 1996 to May ....

A. B. Downey and D. G. Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev. 26(4), pp. 14-29, Mar 1999.

....and thus investigate how scheduling solutions behave under different load conditions. By multiplying both the arrival rate and the request time we keep the characteristics of the workload. In particular, this approach does not shorten the daylong arrival cycle, as does compressing the arrival time [34] [44] Moldability Model The moldability model generates v requests for a job j with one known request (partition size n, request time tr, and accuracy a) That is, the moldability model produces the v tuples n = n ) tr = tr ) a = a ) that describe v 53 requests for job j. ....

....model on all jobs generated by the rigid workload model. That is the focus of Chapter 6. 55 4. Performance Metrics for SA The metric used to compare competing solutions is a key aspect of performance evaluation. Since the use of inappropriate metrics can result in misleading conclusions [34] [44] one wants to find a metric that is unbiased and that captures our intuition of good performance for the target scenario. SA aims to improve the performance of one job. Therefore, our performance metric should capture our intuitive notion of individual job performance. This chapter ....

[Article contains additional citation context not shown here]

Allen Downey and Dror Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev. 26(4), pp. 14-29, March 1999.

....distribution fitting, parameter correlation, and validation) we refer the reader to [5] and [6] 2.3. Performance Metrics The metric used to compare competing solutions is a key aspect of performance evaluation. Since the use of inappropriate metrics can result in misleading conclusions [11] [15] one 11 wants to find metrics that are unbiased and that capture our intuition of good performance for the research scenario. SA aims to improve job performance. Therefore, we need a way to capture our intuitive notion of individual job performance. This goal is fulfilled by the job ....

....slowdown combine the turn around times of all jobs in the evaluation workload into a single value, and hence should be considered as the performance metric to be used here. As we shall see, however, these metrics are not appropriate for our research scenario because they bias towards long jobs [11] [15] and or reward performance poor scheduling strategies for moldable jobs. We then argue that the geometric mean of turn around times is an appropriate performance metric for our research scenario. Mean Turn Around Time Since turn around time provides a good metric for a single job, many ....

[Article contains additional citation context not shown here]

A. Downey and D. Feitelson. "The elusive goal of workload characterization". Perf. Eval. Review 26(4), pg. 14-29, March 1999.

....capacity a throughput benchmark that reflects the needs of the present and projected workload is mandatory. Because such a workload will differ from site to site it is not sensible to develop one static benchmark that should be run by every institution where a major replacement is due[1]. However, a framework can be provided in which the appropriate programs can be inserted and that greatly eases the burden of developing such a benchmark from scratch. The EuroBen Throughput Benchmark Framework presented here tries to make putting together a complete throughput benchmark for ....

A.B. Downey, D.G. Feitelson, The Elusive Goal of Workload Characterization, ACM Sigmetrics, 26, No.4, May 1999, 14--29.

....A software daemon (JMD) developed at NCSA records the actual resource usage of each job every 30 seconds during its execution. The memory and cpu usage of each job is obtained from the JMD log. 2. 2 Related Work Several workload studies [FeNi95, Feit96, Hoto96, HSO96, SGS96, WLF 96, WMKS96, DoFe99] report the measured distributions of the number of requested processors and actual job runtime, on various production systems (e.g. NASA Ames Intel iPSC 860, Argonne SP 1, Cornell Theory Center (CTC) SP 2, SDSC Intel Paragon, PSC T3D) Several of these studies also report the distribution of ....

.... Cornell Theory Center (CTC) SP 2, SDSC Intel Paragon, PSC T3D) Several of these studies also report the distribution of job interarrival time [FeNi95, HSO96, WLF 96] and the relationship between the average or distribution of runtime and requested number of processors [FeNi95, Hoto96, HSO96, DoFe99] The studies in [Feit97, SSN99] focus on the memory usage of jobs on the LANL CM 5 and SDSC CRAY T3E. Feit97] also reports the distribution of the fraction of requested memory used by a job. HSO96] reports the distribution of requested memory per node on the CTC SP 2. Based on job traces from ....

[Article contains additional citation context not shown here]

A. B. Downey and D. G. Feitelson. "The Elusive Goal of Workload Characterization". Performance Eval. Rev., 26(4):14--29, March 1999.

....correlation) as well as for a more detailed validation, we refer the reader to [6] 7] and [8] 4.3. Performance Metrics The metric used to compare competing solutions is a key aspect of performance evaluation. Since the use of inappropriate metrics can result in misleading conclusions [13] [17] one wants to find a metric that is unbiased and that captures our intuition of good performance for the target scenario. SA aims to improve the performance of one job. Therefore, we need a way to capture our intuitive notion of individual job performance. This goal is fulfilled by the job ....

....slowdown combine the turn around times of all jobs in the evaluation workload into a single value, and hence should be considered as the performance metric to be used here. As we shall see, however, these metrics are not appropriate for our research scenario because they bias towards long jobs [13] [17] and or reward performance poor scheduling strategies 1 Turn around time is also referred to as response time. 8 for moldable jobs. We then argue that the geometric mean of turn around times is an appropriate performance metric for our research scenario. Mean Turn Around Time Since ....

[Article contains additional citation context not shown here]

A. Downey and D. Feitelson. The elusive goal of workload characterization. Performance Evaluation Review 26(4), pg. 14-29, March 1999. http://www.cs.huji.ac.il/~feit/pub.html

....Downey s # Normally distributed # = 1.209; # # = 1.132 Table 3 Summary of the Moldability Model 2.3. Performance Metrics The metric used to compare competing solutions is a key aspect of performance evaluation. Since the use of inappropriate metrics can result in misleading conclusions [12] [15] one wants to find a metric that is unbiased and that captures our intuition of good performance for the target scenario. SA aims to improve job performance. Therefore, we need a way to capture our intuitive notion of individual job performance. This goal is fulfilled by the job turn around ....

.... combine the turn around times of all jobs in the evaluation workload into a single value, and hence should be considered as the performance metric to be used here. As we shall 11 see, however, these metrics are not appropriate for our research scenario because they bias towards long jobs [12] [15] and or reward performance poor scheduling strategies for moldable jobs. We then argue that the geometric mean of turn around times is an appropriate performance metric for our research scenario. Mean Turn Around Time Since turn around time provides a good metric for a single job, many ....

[Article contains additional citation context not shown here]

Allen Downey and Dror Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev. 26(4), pp. 14-29, March 1999. http://www.cs.huji.ac.il/~feit/pub.html

....Downey s # Normally distributed # = 1.209; # # = 1.132 Table 3 Summary of the Moldability Model 3.3. Performance Metrics The metric used to compare competing solutions is a key aspect of performance evaluation. Since the use of inappropriate metrics can result in misleading conclusions [12] [15] one wants to find a metric that is unbiased and that captures our intuition of good performance for the target scenario. SA aims to improve the performance of one job. Therefore, we need a way to capture our intuitive notion of individual job performance. This goal is fulfilled by the job ....

....into a single value, and hence should be considered as the performance 1 Turn around time is also referred to as service time or response time. 9 metric to be used here. As we shall see, however, these metrics are not appropriate for our research scenario because they bias towards long jobs [12] [15] and or reward performance poor scheduling strategies for moldable jobs. We then argue that the geometric mean of turn around times is an appropriate performance metric for our research scenario. Mean Turn Around Time Since turn around time provides a good metric for a single job, many ....

[Article contains additional citation context not shown here]

A. Downey and D. Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev. 26(4), pp. 14-29, March 1999. http://www.cs.huji.ac.il/~feit/pub.html

....by applying our moldable job model to the jobs in the rigid workload. The source of the rigid workload is not important for our model. It can be a workload log (such as the those described in Table 1 and elsewhere [11] 15] 16] 18] or originate from a model for rigid workloads [2] 5] 8] [9] [12] 14] 17] In order to be able to build a realistic model, we designed a user survey to investigate the characteristics moldable jobs exhibit in practice. The survey consisted of 12 multiple choice questions, and was conducted online via email and the Web between 17 April and 31 May 2000. ....

....parameters A and s (as we shall see in Section 4) and hence does not need to be modeled directly. 3.1. Partition Size Constraints As Table 1 shows, the distribution of partition sizes are dominated by power of 2 values in many workload logs. This phenomenon was also observed by others [7] [9] [13] 20] There has been some controversy as to whether to incorporate the dominance of power of 2 partitions into a workload model. Some researchers have accounted for the high incidence of power of 2 jobs and modeled the partition size accordingly [13] Others, however, believe that this ....

[Article contains additional citation context not shown here]

Allen Downey and Dror Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev. 26(4), pp. 14-29, March 1999. http://www.cs.huji.ac.il/~feit/pub.html

....representative workload. The choice of workload is crucial, as di erent workloads can lead to very di erent performance results. It seems that ESP has made a very reasonable choice in this respect. The proposed workload conforms to various features of workloads observed in production installations [4], including 72 The distribution of job sizes, which is mostly powers of two, but not only The existence of a large variance in the distribution of runtimes The repetition of certain jobs The size of the test 82 jobs is also a reasonable compromise between the desire to have enough ....

A. B. Downey and D. G. Feitelson, \The elusive goal of workload characterization". Perf. Eval. Rev. 26(4), pp. 14-29, Mar 1999.

.... 18, 6] Other examples include studies of process arrivals and runtimes [12, 37] le systems [36] and video streams [48] In the area of parallel systems, descriptive studies of workloads have only started to appear in recent years [29, 76, 58, 27, 14] There are also some attempts at modeling [10, 28, 21, 41, 23, 54, 15] and on line characterization [34] But where does the data come from There are two main options: use data that is available anyway, or collect data speci cally for the workload model. The latter can be done in two ways: active or passive instrumentation. Importantly, collected data can and ....

....0.8 1 Feitelson model serial 2 4 5 8 9 32 32 Fig. 1. Distributions of runtimes for di erent ranges of job sizes, in two workload logs and two models of parallel jobs. One way to select suitable distributions is based on moments, and especially the mean and the variance of the sample data [23]. For example, these statistics indicate that the distribution of job runtimes has a wide dispersion, leading to a preference for a hyper exponential model over an exponential one. Jann et al. 9 have used hyper Erlang distributions to create models that match the rst 3 moments of a distribution ....

[Article contains additional citation context not shown here]

A. B. Downey and D. G. Feitelson, \The elusive goal of workload characterization". Performance Evaluation Rev. 26(4), pp. 14-29, Mar 1999.

....(e.g. 24 hours) 4.4 The Alternative: Estimates Based on Historical Information It is well known that the workload on parallel supercomputers is highly repetitive. This means that the same users tend to run the same programs over and over again, sometimes up to hundreds of executions in a row [8, 3]. It stands to reason that such repeated executions of the same application would have highly correlated runtimes, and indeed several studies have shown that it is possible to derive crude estimates of runtimes using such information [10, 2, 24] However, these studies were done in a context that ....

A. B. Downey and D. G. Feitelson, \The elusive goal of workload characterization". Perf. Eval. Rev. 26(4), pp. 14-29, Mar 1999.

....24 hours) 4.4 The Alternative: Estimates Based on Historical Information It is well known that the workload on parallel supercomputers is highly repetitive. This means that the same users tend to run the same programs over and over again, sometimes up to hundreds of executions in a row [8] [3]. It stands to reason that such repeated executions of the same application would have highly correlated runtimes and, indeed, several studies have shown that it is possible to derive crude estimates of runtimes using such informa tion [10] 2] 24] However, these studies were done in a ....

A.B. Downey and D.G. Feitelson, "The Elusive Goal of Workload Characterization," Performance Evaluation Review, vol. 26, no. 4, pp. 14-29, Mar. 1999.

....logs contain the details of all jobs run on the system, including their arrival, start, and end times, the number of processors they used, the amount of memory used, the user who ran the job, the executable le name, etc. By analyzing this data, a statistical model of the workload can be created [7, 9]. This should focus on recurrent features that appear in logs derived from di erent installations. At the same time, features that are inconsistent at di erent installations should also be identi ed, so that their importance can be veri ed. A good example is the rst such analysis, published in ....

....we can compare the runtime statistics of jobs that use di erent numbers of nodes. the result is that there is little if any correlation in the statistical sense. However, the distributions of runtimes for small and large jobs do tend to be di erent, with large jobs often having longer runtimes [7] (Figure 3) This favors the memory bound or xed time scaling models, and contradicts the xed work model. There is also some evidence that larger jobs use more memory [10] Thus, within a sin 8 gle machine, parallelism is in general not used for speedup but for solving larger problems. Direct ....

A. B. Downey and D. G. Feitelson, \The elusive goal of workload characterization". Performance Evaluation Rev. 26(4), pp. 14-29, Mar 1999.

....number of parameters (about 40) that closely mimics the original data. One problem with creating distributions based on moments is that with skewed distributions the estimation of high order moments (and even the second moment) is very sensitive to the values of the few highest values sampled [6]. This has lead to the proposed use of distributions based on direct observations of the CDF and goodness of fit metrics, in lieu of trying to match the moments. Feitelson used a two stage or three stage hyperexponential distribution, choosing the parameters so that the CDF looked right (that ....

A. B. Downey and D. G. Feitelson, "The elusive goal of workload characterization ". Performance Evaluation Rev. 26(4), pp. 14--29, Mar 1999.

....correctly. Indeed, more than 20 years ago Lazowska showed that models based on a hyperexponential distribution with matching moments can lead to incorrect results [15] In addition, the calculation of moments can be unduly in uenced by extreme values, that are not necessarily representative [4]. Thus, we prefer the calculation of percentiles and an attempt to match the CDF of the target distribution. In Section 3, we present our techniques for modeling the CDF of a distribution. Our chosen model is the hyper Gamma distribution, which is a combination of two Gamma distributions. The ....

....moldable jobs, it is necessary to model the total work and the speedup function, in order to derive the runtime for any allocated number of processors. 4. 1 Handling Power of Two Jobs Experience shows that the distribution of job sizes is typically dominated by sizes that are powers of two [4]. This is obvious for early machines that only allowed powers of two, such as hypercubes and connection machines. However, it persists also in systems that have no architectural preference for powers of two. This raises the question of what is the source of this phenomenon, and whether it should ....

[Article contains additional citation context not shown here]

A. B. Downey and D. G. Feitelson, \The elusive goal of workload characterization". Performance Evaluation Rev. 26(4), pp. 14-29, Mar 1999.

....design and implementation, but also on the workload to which they are subjected [13, 1] Thus performance evaluations must use a representative workload, be it a direct trace from an existing system, or a mathematical model. Modeling workloads, in particular, has several attractive attributes [7]. It increases our understanding of the interactions between the computer system and the workload. It fosters more reliable evaluations by allowing for the creation of similar workloads and the calculation of con dence intervals. It enables making reasonable changes in workload parameters in order ....

....number of parameters (about 40) that closely mimics the original data. One problem with creating distributions based on moments is that with skewed distributions the estimation of high order moments (and even the second moment) is very sensitive to the values of the few highest values sampled [7]. This has lead to the proposed use of distributions based on direct observations of the CDF and goodness of t metrics, in lieu of trying to match the moments. Feitelson used a two stage or three stage hyperexponential distribution, choosing the parameters so that the CDF looked right (that ....

A. B. Downey and D. G. Feitelson, \The elusive goal of workload characterization". Perf. Eval. Rev. 26(4), pp. 14-29, Mar 1999.

....(e.g. 24 hours) 4.4 The Alternative: Estimates Based on Historical Information It is well known that the workload on parallel supercomputers is highly repetitive. This means that the same users tend to run the same programs over and over again, sometimes up to hundreds of executions in a row [8, 3]. It stands to reason that such repeated executions of the same application would have highly correlated runtimes, and indeed several studies have shown that it is possible to derive crude estimates of runtimes using such information [10, 2, 24] However, these studies were done in a context that ....

A. B. Downey and D. G. Feitelson, \The elusive goal of workload characterization". Perf. Eval. Rev. 26(4), pp. 14-29, Mar 1999.

No context found.

A. B. Downey and D. G. Feitelson. The Elusive Goal of Workload Characterization. Technical report, Hebrew University, Jerusalem, March 1999.

No context found.

Allen Downey and Dror Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev. 26(4), pp. 14-29, March 1999. http://www.cs.huji.ac.il/~feit/pub.html

No context found.

A. Downey and D. Feitelson. The elusive goal of workload characterization. Performance Evaluation Review, 26(4):14--29, 1999.

No context found.

Allen B. Downey and Dror G. Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev., 26(4):14--29, Mar 1999.

No context found.

Allen B. Downey and Dror G. Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev., 26(4):14--29, Mar 1999.

No context found.

A. B. Downey and D. G. Feitelson. The elusive goal of workload characterization. Perf. Eval. Rev., 26(4): 14--29, 1999.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC