This chapter presents principles and techniques for modelbased black-box conformance testing of real-time systems using the Uppaal model-checking tool-suite. The basis for testing is given as a network of concurrent timed automata specified by the test engineer. Relativized input/output conformance serves as the notion of implementation correctness, essentially timed trace inclusion taking environment assumptions into account. Test cases can be generated offline and later executed, or they can be generated and executed online. For both approaches this chapter discusses how to specify test objectives, derive test sequences, apply these to the system under test, and assign a verdict.

ix 0.1 An Overview of Software Testing Methods # # # # # # # # # # # # # # # 2 0.2 Provable Security and Formal Methods # # # # # # # # # # # # # # # # # 9 0.3 Security Testing # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 10 0.4 Applications of Fault Categories # # # # # # # # # # # # # # # # # # # # # 11 0.5 Organization of the Thesis # # # # # # # # # # # # # # # # # # # # # # # # 12 1. RELATED WORK # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 14 1.1 Protection Analysis Project # # # # # # # # # # # # # # # # # # # # # # # 14 1.2 RISOS Project # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 19 1.3 Flaw Hypothesis Methodology # # # # # # # # # # # # # # # # # # # # # # 21 1.4 Case Study# Penetration Analysis of the Michigan Terminal System # 23 1.5 Software Fault Studies # # # # # # # # # # # # # # # # # # # # # # # # # # 25 1.5.1 Fault Categories # # # # # # # # # # # # # # # # # # # # # # # # # # 27 1.6 Errors of T E X # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 31 1.7 A Taxonomy of Computer Program Security Flaws # # # # # # # # # # 32 1.8 Comparison of Security Fault Classi#cation Schemes # # # # # # # # # # 33 2. A TAXONOMY OF SECURITY FAULTS IN THE UNIX OPERATING SYSTEM # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 35 2.1 A Taxonomy of Security Faults # # # # # # # # # # # # # # # # # # # # # 36 2.2 Con#guration Errors # # # # # # # # # # # # # # # # # # # # # # # # # # # 40 2.2.1 Examples # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 40 2.3 Synchronization Errors # # # # # # # # # # # # # # # # # # # # # # # # # # 41 2.3.1 Example # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 41...

This paper argues that with the advent of explicitly speci ed software architectures, testing can be done e ectively at the architectural level. A software architecture speci-cation provides a solid foundation for developing a plan for testing at this level. We propose several architecture-based test criteria based on the Chemical Abstract Machine model of software architecture. An architectural (integration) test plan, developed by applying selected of these criteria, can be used to assess the architecture itself or to test the implementation's conformance with the architecture. This facilitates detecting defects earlier in the software lifecycle, enables leveraging software testing costs across multiple systems developed from the same architecture, and also leverages the e ort put into developing a software architecture. 1

A discontinuity exists between object-oriented modeling and programming languages. This discontinuity arises from ambiguous concepts in modeling languages and a lack of corresponding concepts in programming languages. It is particularly acute for binary class relationships -- association, aggregation, and composition. It hinders the traceability between software implementation and design, thus hampering software analysis. We propose consensual definitions of the binary class relationships with four minimal properties -- exclusivity, invocation site, lifetime, and multiplicity. We describe algorithms to detect automatically these properties in source code and apply these on several frameworks. Thus, we bridge the gap between implementation and design for the binary class relationships, easing software analysis.

Paths can reveal a program’s dynamic behavior and uncover patterns of path locality that can be exploited to increase program performance. The authors explore several methods for doing so.

Size and code coverage are two important attributes that characterize a set of tests. When a program P is executed...

Exception-handling constructs provide a mechanism for raising exceptions, and a facility for designating protected code by attaching exception handlers to blocks of code. Despite the frequency of their occurrences, the behavior of exception-handling constructs is often the least understood and poorly tested part of a program. The presence of such constructs introduces new structural elements, such as control-flow paths, in a program. To adequately test such programs, these new structural elements must be considered for coverage during structural testing. In this paper, we describe a class of adequacy criteria that can be used to test the behavior of exception-handling constructs. We present a subsumption hierarchy of the criteria, and illustrate the relationship of the criteria to those found in traditional subsumption hierarchies. We describe techniques for generating the test requirements for the criteria using our control-flow representations. We also describe a methodology f...

"Software test-coverage measures" quantify the degree of thoroughness of testing. Tools are now available that measure test-coverage in terms of blocks, branches, computation-uses, predicate-uses, etc. that are covered. This paper models the relations among testing time, coverage, and reliability. An LE (logarithmic -exponential) model is presented that relates testing effort to test coverage (block, branch, computation-use, or predicate -use). The model is based on the hypothesis that the enumerable elements (like branches or blocks) for any coverage measure have various probabilities of being exercised; just like defects have various probabilities of being encountered. This model allows relating a test-coverage measure directly with defect-coverage. The model is fitted to 4 data-sets for programs with real defects. In the model, defect coverage can predict the time to next failure. The LE

Software maintainers are faced with the task of regression testing: retesting a modified program on a (large) number of test cases. The cost of regression testing can be reduced if old test cases and old test results can be reused. Reuse avoids the costly construction of new test cases and the unproductive rerunning of existing test cases when it can be guaranteed that the modified and original programs will produce the same results. An algorithm that uses language semantics to provide such a guarantee is presented. This algorithm uses semantic (not syntactic) differences and similarities between the old and new programs. The algorithm is based on the notion of common execution patterns, which is the interprocedural extension of equivalent execution patterns. Program components with common execution patterns are computed using a new type of interprocedural slice called a calling context slice. Whereas an interprocedural slice includes the program components necessary to capture all pos...

The goal of software testing analysis is to validate that an implementation satisfies its specifications. Many errors in software are caused by generalizable flaws in the source code. Property-based testing assures that a given program is free of the specified generic flaws. Property-based testing uses property specifications and a data-flow analysis of the program to guide evaluation of test executions for correctness and completeness.