David A. Bailey. Misleading performance reporting in the supercomputing eld. Scientic Programming, 1(2):141-151, 1992. 155 BIBLIOGRAPHY 156  Home/Search   Document Details and Download   Summary   Related Articles   Check

....awed interpretation of slot ll ratios is tempting to make optimising transformations for ILP code look better than they are and have been found by the author in di erent publications. This is a point to consider in the accurate presentation of scienti c results, as proposed for example by Bailey [10] or Crowl [20] CHAPTER 5. HYBRIDISING GAPS 101 5.7 Improving Scalability 5.7.1 Improving Scalability Using Analysis and Pro ling For the future prospects of GAPS it has to be able to handle large programs. However, the study on the size of the transformation space conducted in Chapter 4 has ....

David A. Bailey. Misleading performance reporting in the supercomputing eld. Scientic Programming, 1(2):141-151, 1992. 155 BIBLIOGRAPHY 156

....parallel execution of multiple processes. Usually, different benchmark program measurements are summarized in order to find the average performance of a computer. How to calculate these averages and how to report them has been one of the most confusing issues in performance evaluation [35] 36] [37] [22] Some of the above stated problems cannot be spirited away entirely, but are deliberately neglected when the performance of a uni processor system is described with a peak rate. This gets even worse in the case of parallel systems, since the performance of such a system is unfortunately not ....

David H. Bailey, "Misleading Performance Reporting in the Supercomputing Field", Tech. Rep. RNR-92-005, Numerical Aerodynamic Simulation (NAS) Systems Division, NASA Ames Research Center, December 1992.

....and on a single processor machine. 1) Peak performance = 66 Mflops. Clearly, it is pointless to employ numerically inefficient algorithms merely to exhibit artificially high performance rates on a particular parallel machine using algorithms which are inefficient on a serial computer, cf [6]) It can be seen that any algorithm with an optimal order of computational complexity O(N) requires some form of global communication such as occurs in the multilevel iteration methods. In Table 2 we consider now the use of the multilevel iteration method on a massively parallel computer. ....

D.H. Bailey (1992). Misleading Performance Reporting in the Supercomputing Field, Scientific Progr. 1, 141--151.

....seems unlikely in the near future. 2.3 Performance measurement The reporting of performance results in the field of high performance computing has acquired something of a poor reputation in recent years, which extends to scientific publications as well material produced by vendors. Bailey [1] gives a useful critique of practices in this area, and sets out a number of guidelines designed to improve standards of performance reporting, to which we attempt to adhere in our experiments. The fundamental metric for the measurement of the performance of a program running on a parallel ....

....the performance of the parallel implementation in terms of the number of flops executed per second. Note that many authors report only flop rates, and neglect to mention the difference in flop counts between sequential and parallel algorithms. This is another example of poor practice cited in [1] and [26] For quadrature problems, flop counts are inappropriate as they depend strongly on the nature of the integrand. A much more suitable unit of execution is the CHAPTER 2. PARALLEL COMPUTING 37 number of integrand evaluations. Thus we can measure the algorithms effectiveness in terms of ....

Bailey, D.H., (1992) Misleading performance reporting in the supercomputing field, Scientific Programming, vol. 1, no. 2, pp. 141--151.

....seems unlikely in the near future. 2.3 Performance measurement The reporting of performance results in the field of high performance computing has acquired something of a poor reputation in recent years, which extends to scientific publications as well material produced by vendors. Bailey [1] gives a useful critique of practices in this area, and sets out a number of guidelines designed to improve standards of performance reporting, to which we attempt to adhere in our experiments. The fundamental metric for the measurement of the performance of a program running on a parallel ....

....the performance of the parallel implementation in terms of the number of flops executed per second. Note that many authors report only flop rates, and neglect to mention the difference in flop counts between sequential and parallel algorithms. This is another example of poor practice cited in [1] and [26] For quadrature problems, flop counts are inappropriate as they depend strongly on the nature of the integrand. A much more suitable unit of execution is the CHAPTER 2. PARALLEL COMPUTING 37 number of integrand evaluations. Thus we can measure the algorithms effectiveness in terms of ....

Bailey, D.H., (1992) Misleading performance reporting in the supercomputing field, Scientific Programming, vol. 1, no. 2, pp. 141--151.

....5 Conclusions Deciding how best to present parallel performance results is a non trivial task. A bad choice of what to measure, or how to present these measurements, can lead to information being hidden or distorted, either accidentally, as discussed in [3] or deliberately, as criticised in [1]. If the initial performance graphs in Figure 1 had been plotted showing only times for 1, 2, 4, 8, etc. processors, then the drop in performance for the 8 Theta 8 problem size would have gone unnoticed. Although potentially misleading, this practice is not mentioned in either [1] or [3] As they ....

....as criticised in [1] If the initial performance graphs in Figure 1 had been plotted showing only times for 1, 2, 4, 8, etc. processors, then the drop in performance for the 8 Theta 8 problem size would have gone unnoticed. Although potentially misleading, this practice is not mentioned in either [1] or [3] As they are, the graphs shown are adequate for conveying intra problemsize performance information, but they allow no inter problem size comparisons. Unfortunately, accurate measurement of absolute performance (i.e. in Mflop s Gamma1 ) is not always possible as hardware counters for ....

Bailey, D. H. (1992) Misleading Performance Reporting in the Supercomputing Field, Scientific Programming, vol. 1, no. 2, pp. 141--151.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC