Work Package 2 Common Interfaces and Integration Techniques
|
Abstract:
Automatic performance tools must of course be tested as to whether they perform their task correctly. Because performance tools are meta-programs, tool testing is more complex than ordinary program testing, and comprises at least three aspects. First, it must be ensured that the tools do neither alter the semantics nor distort the runtime behavior of the application under investigation. Next, it must be verified that the tools collect the correct performance data as required by their specification. Finally, it must be checked that the tools indeed perform their intended tasks: For badly performing applications, relevant performance problems must be detected and reported, and, on the other hand, tools should not diagnose performance problems for well-tuned programs without such problems. In short, performance tools should be semantics-preserving, complete and correct. Focusing on the correctness aspect, testing can be done by using synthetic test functions with controllable performance properties, and/or real world applications with known performance behavior. A systematic test suite can be built from such functions and other components, possibly with the help of tools to assist the user in putting the pieces together into executable test programs. Clearly, such a test suite can be highly useful to builders of performance analysis tools, such as the members of the APART working group, who currently pursue a variety of tool development projects. It is surprising that
