Gargantini, A. and Heitmeyer, C. (1999). Using model checking to generate tests from requirements specifications. In Proceedings of the 7th European Software Engineering Conference, Held Jointly with the 7th ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE'99, volume 1687 of LNCS, pages 146--162. Springer-Verlag.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....our objective was achieved by a single trace, the simulation was generated completely automatically, requiring no user input. 2) Test case generation: Although the primary goal of model checking is to verify a model against temporal properties, it has recently been used to generate test cases [27], 28] 29] 26] Most of the proposed techniques are based on the fact that in addition to computing values of the given properties, a model checker can produce witnesses (or counterexamples) which can be used to construct test sequences. The properties that are used to force the model checker ....

....on the fact that in addition to computing values of the given properties, a model checker can produce witnesses (or counterexamples) which can be used to construct test sequences. The properties that are used to force the model checker to generate desired test sequences are called trap properties [27]. Gargantini and Heitmeyer [27] proposed a method that uses an SCR specification of a system to identify trap properties satisfying a form of branch coverage testing criterion. Their technique uses both mode transition and condition tables to generate test sequences. We illustrate the method on ....

[Article contains additional citation context not shown here]

A. Gargantini and C. Heitmeyer, "Using Model Checking to Generate Tests from Requirements Specifications," in Proceedings of Joint 7th European Software Engineering Conference and 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE99). Toulouse, France: ACM Press, September 1999, pp. 146--162.

....the bootstrap strategy with real software testing developments is a crucial issue. This comparison should begin by an exhaustive analysis of the current testing methods in the light of the epistemological framework here presented. A starting point could be, for instance, the method reported in [6], whose similarity with this strategy was pointed out by C. Heitmeyer. ....

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In O. Nierstrasz and M. Lemoine, editors, Proc. of ESEC/FSE'99, volume 1687 of LNCS. Springer-Verlag, May 1999.

....modelchecking techniques. In addition, recent research expanded the potential of the use of model checking in requirements engineering beyond analysing classical state machine models. We now have techniques for reasoning about high level goals [19] use of the modelchecker for test case generation [20,21], formal support for analysing incomplete and inconsistent models [22,23] and the use of the model checker for general model exploration via query checking [24] Case studies and further research would be necessary to determine which of these applications of model checking technology will really ....

Gargantini A, Heitmeyer C. Using model checking to generate tests from requirements specifications. In: Proceedings of the joint 7th European software engineering conference and 7th ACM SIGSOFT international symposium on foundations of software engineering (ESEC/FSE99), Toulouse, France, September 1999. ACM Press, New York, pp 146--162

....generation problem is formulated as finding counterexamples during Figure 1. Exemplary Resource Distribution on Different SQA Methods the model checking [9] 14] Another similar approach is to model the negation of certain properties to be covered and generate the counterexamples as test cases [12]. Application of mutation analysis in model checking can generate counterexamples as test cases to distinguish the variants from original specification [1] TestEra [20] models the correctness criteria for the program in Alloy, a first order relational language, and specifies abstraction and ....

A. Gargantini, and C. Heitmeyer, Using Model Checking to Generate Tests from Requirements Specifications, ESEC/FSE '99, Toulose, France, September 1999, Lecture Notes in Computer Science, Vol. 1687.

....we can now precisely scale the test process, justifying the cumulating costs that must be in compliance with the test budget. Beside L. White s pioneer work, another state oriented approach, based on the traditional method SCR (Software Cost Reduction) is described by C. Heitmeyer et al. in [GARG]. This approach uses model checking to generate test cases, using well known coverage metrics for test case selection. For expressing conditioned events in temporal logic formulae, the authors propose to use modal logic abbreviations which requires some skill with this kind of formalism. A ....

A. Gargantini, C. Heitmeyer, "Using Model Checking to Generate Tests from Requirements Specification ", Proc. ESEC/FSE '99, ACM SIGSOFT, pp. 146-162, 1999

....correctness, the model checker reports counterexample(s) that violate a correctness criterion. While generating test inputs, the model checker generates paths that are witnesses to a testing criterion encoded in the speci cation. Testing criteria can be encoded as correctness speci cations as in [8,13]. For every reported path, the model checker also reports the input heap con guration, the path condition for the primitive input elds thread scheduling, which can be used to reproduce the error. Multi threaded and non deterministic systems Our framework allows a standard model checker to ....

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements speci cations. In Proc. 7th European Engineering Conference held jointly with the 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 146-162. Springer-Verlag, 1999.

.... for a safety injection control system [Cou93] Although fairly small, this case study comes from a real application, raises many of the issues found in high assurance systems and is frequently used to illustrate other methods such as, e.g. the SCR method [Heit96] and its analysis techniques [Bha99, Jef98, Gar99]. Illustrations on larger, more complex systems such as the LAS ambulance despatching system and the BART train control system can be found in [Lam00a, Lam00b, Let01] 2. GOAL ORIENTED ANALYSIS OF REQUIRE MENTS FOR A SAFETY INJECTION SYSTEM We follow the KAOS method [Lam00b] to gradually ....

A. Gargantini and C. Heitmeyer, "Using Model Checking to Generate Tests from Requirements Specifications", Proc., ESEC'99 & 7th ACM S1GSOFT Intern. Symp. on Foundations of Software Eng. (ESEC/FSE99), Toulouse, September 1999.

....test suite finds specific classes of failures [40] mutations could apply to both a specification and an implementation. One could either generate a test suite manually to kill all mutants or use a technique (such as modelchecking) to identify differences between an original and a mutated model [1, 21]. At the same time, one could derive test suites from a model in order to demonstrate behavioural equivalence of an implementation to this model by testing, for instance, using well known automata theoretic results [14, 46, 47] and their extensions to X machines (EFSM) 30, 32] and Statecharts [5, ....

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In Proc. ESEC/FSE 99: Joint 7th European Software Engineering Conference (ESEC) and 7th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE-7), Toulouse, France, 6--10 September 1999.

.... use the Simple PROMELA Interpreter (SPIN) model checker [16] to generate tests that cover each block in a certain partitioning of the input domain [8] Gargantini and Heitmeyer use both SPIN and SMV to generate branch adequate tests from Software Cost Reduction (SCR) requirements speci cations [14]. The model checking approach to formal methods has received considerable attention in the literature, and readily available tools such as SMV and SPIN are capable of handling the state spaces associated with realistic problems [11] Although model checking began as a method for verifying ....

Angelo Gargantini and Constance Heitmeyer. Using model checking to generate tests from requirements speci cations. In Proceedings of the Joint 7th European Software Engineering Conference and 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, Toulouse, France, September 1999. To Appear.

....to a debugging or fault finding mode in which the analyst is seeking the cause of the problem. Finite state verification approaches usually provide a trace or path through the model (or the corresponding path through the original software system) but these paths are sometimes long and convoluted [2, 9]. Complicated paths make it more difficult to track down the actual cause of the inconsistency. Thus, when an analyst is in fault finding mode, it makes sense to use a reasoning engine optimized to produce short paths or user guided paths that reveal the inconsistency. A more mature software ....

....algorithms for performing points to analysis [11, 14] The problem of comparing verification based algorithms or of finding counter examples through a data flow analysis problem does not seem to have been studied. Some work has been done on counter example generation for testing [1, 3] In [9], model checking is used to create counter examples for the inverse of a property. This returns traces that are paths over which the property holds. These traces are then used to select tests for the system. In this work, two different model checkers, SMV [16] and SPIN [12] were used to create ....

[Article contains additional citation context not shown here]

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In Proc. of the Seventh European Soft. Eng. Conf. held jointly with the Seventh ACM SIGSOFT Symp. on the Found. of Soft. Eng., pages 146--162, Sept. 1999.

.... been extensive research in using formal methods to verify security critical systems, mostly with respect to secure information ow [GM82, O H90] or security protocols (e.g. BAN89, Low96] an overview is in [RSG 01] Work on speci cation based testing has been presented e.g. in [FS99, RR99, GH99, WLPS00, LMW01] An application to security ( rewall testing) is given e.g. in [JW01b] Formal methods have already been applied to smart card security, e.g. in [And99, BCG 00, BCM 00] J ur01a] uses the Uni ed Modeling Language to reason about part of CEPS, but without considering ....

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specications. In Proc., Joint 7th Eur. Software Engineering Conf. and 7th ACM SIGSOFT Intern. Symp. on Foundations of Software Eng. (ESEC/FSE99), 1999.

....they used the term negative test for what we call a failing test. Ammann et al. defined a mutation analysis approach to generating and recognizing tests with a model checker [2, 3] Gargantini and Heitmeyer used model checkers to generate tests for systems with SCR requirement specifications [17]. Their method yields branch coverage on the SMV description of the requirements; the relationship between branch and mutation coverage at the specification level is essentially the same as it is at the program source code level. Ritchey and Ammann used a model checker to provide comprehensive ....

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In Proceedings of the 7th European Software Engineering Conference held joinly with the 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 146--162, Toulouse, France, September 1999.

....mode classes, whose values are modes, and terms. SCR specifications are organized in event tables (Example 26) and condition tables: the former identify all events the system is sensitive to; the latter define system responses. Example 26 The event table that defines the mode class Pressure [GH99] The Safety Injection System (SIS) is a simplified system for safety injection in a nuclear plant. Its inputs are the monitored variables waterPres, block, and reset, and the single output is the controlled variable safetyInjection. The specification includes also a mode class pressure whose ....

....initial state and leading to a state in which # is true, which is a suitable state for testing the implication # # # . c) The trace is used to generate a test sequence by executing it on the automaton that represents the system (Example 27. Example 27 An example of test sequence generation [GH99] Suppose the following property: T(waterPres # low) WHEN block=On # reset=Off # safetyInjection =Off 24 In Section 4.2 we have already discussed interpreting tabular SCR interface specifications as oracles. We grouped that approach with others that directly described program ....

[Article contains additional citation context not shown here]

Angelo Gargantini and Connie Heitmeyer. Using Model Checking to Generate Tests from Requirements Specifications. In Proceedings of the 7th European Engineering Conference and the 7th ACM SIGSOFT Symposium on the Foundations of Software Engeneering, volume 24.6 of Software Engineering Notes (SEN), pages 146--162. ACM Press, September 6--10 1999.

....contexts. Mutation analysis is used in the approach of [1] In [6] test generation is performed from user specified temporal formulas, while in [10] testing purposes are used to generate tests. No consideration is given to coverage criteria. Some control flow coverage criteria are considered in [11]. We are not aware of any work that considers the model checking approach to data flow oriented test generation. Organization of the paper. Section 2 reviews statecharts and CTL model checking. Section 3 gives a formal definition of the STATEMATE semantics. Section 4 introduces several notions ....

A. Gargantini and C. Heitmeyer, "Using Model Checking to Generate Tests from Requirements Specifications, " in Proceedings of the Joint 7th European Software Engineering Conference and 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 6-10, 1999.

.... began as a method for verifying hardware designs, there is growing evidence that it can be applied to specifications for large software systems, such as TCAS II [7] In addition to verifying properties of software, model checking is being applied to test generation and test coverage evaluation [2, 6, 10, 11]. In both uses, one begins with selection of a test criterion [12] that is, a decision about what properties of a specification must be exercised to constitute a thorough test. Some specification based test criteria are conjunctive com plementary closure partitions [6] branch coverage [11] ....

....10, 11] In both uses, one begins with selection of a test criterion [12] that is, a decision about what properties of a specification must be exercised to constitute a thorough test. Some specification based test criteria are conjunctive com plementary closure partitions [6] branch coverage [11], and mutation adequacy [1] The chosen test criterion is applied to the specification to derive test requirements, i.e. a set of individual properties to be tested. To use a model checker, these requirements are represented as temporal logic formulas. To evaluate coverage of a test set, each ....

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In Proceedings of the Joint 7th European Software Engineering Conference and 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, Toulouse, France, September 1999.

....to importance of model checking applications. The main area of model checking applications is automatic verification of hard and software presented as finite state systems [11] while automatic model checking verification of high level software specifications [7, 1] or automatic test generation [18] are rapidly developing new application domains. We suppose that in both cases high level reliability of model checkers is of extreme importance due to automatic character of model checking. But in spite of importance of reliability issues of verification tools, there are weak moves only in the ....

Gargantini A., Heitmeyer C. Using Model Checking to Generate Tests from Requirements Specifications. ACM SIGSOFT Software Engineering Notes, v.24, n.6, 1999, p.146-162 (simultaneously Lecture Notes in Computer Science, v.1687, 1999)

....of model checking applications. The main area of a model checking applications is automatic verification of hard and software presented as finite state systems [20] while an automatic model checking verification of high level software specifications [21, 22] or automatic test generation [23] are rapidly developing new application domains. We suppose that in both cases a high level reliability of model checkers is of extreme importance due to an automatic character of model checking. But in spite of importance of reliability issues of verification tools there are weak moves only in ....

Gargantini A., Heitmeyer C. Using Model Checking to Generate Tests from Requirements Specifications. ACM SIGSOFT Software Engineering Notes, v.24, n.6, 1999, p.146-162 (simultaneously Lecture Notes in Computer Science, v.1687, 1999)

....ran them against a target implementation, and measured code branch coverage. Generating tests to kill all mutants is the rst test criterion we investigated. Some other speci cation based criteria are stuck at faults [1] CCC partitions [6] MC DC [7] automata theoretic [8] branch coverage [10], disconnection or redirection faults [11] and transition pair coverage [13] Test generation then is the problem of nding tests which ful ll the goals embodied in the criterion. Test set recognition is the conjugate of test generation. Whereas test generation asks, What tests will satisfy the ....

....root access on machine X. If the con guration in fact allows such access given the set of known attacks, a counterexample is produced illustrating the attack. In work underway, we encoded di erent test criteria as temporal logic constraints. Using a model checker, we analyzed branch coverage [10], uncorrelated full predicate coverage (similar to Multiple Condition Decision Coverage or MC DC [7] and transition pair coverage [13] in addition to mutation coverage. We found that di erent metrics are easily encoded into temporal logic, with some limitations, and that interesting theoretical ....

[Article contains additional citation context not shown here]

Angelo Gargantini and Constance Heitmeyer. Using model checking to generate tests from requirements specications. In Proceedings of the Joint 7th European Software Engineering Conference and 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, Toulouse, France, September 1999. To Appear.

.... began as a method for verifying hardware designs, there is growing evidence that it can be applied to specifications for large software systems, such as TCAS II [7] In addition to verifying properties of software, model checking is being applied to test generation and test coverage evaluation [2, 6, 10, 11]. In both uses, one begins with selection of a test criterion [12] that is, a decision about what properties of a specification must be exercised to constitute a thorough test. Some specification based test criteria are conjunctive com plementary closure partitions [6] branch coverage [11] ....

....10, 11] In both uses, one begins with selection of a test criterion [12] that is, a decision about what properties of a specification must be exercised to constitute a thorough test. Some specification based test criteria are conjunctive com plementary closure partitions [6] branch coverage [11], and mutation adequacy [1] The chosen test criterion is applied to the specification to derive test requirements, i.e. a set of individual properties to be tested. To use a model checker, these requirements are represented as temporal logic formulas. To evaluate coverage of a test set, each ....

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In Proceedings of the Joint 7th European Software Engineering Conference and 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, Toulouse, France, September 1999.

....case explicit, and minimizing expressions are given in [2] The set of clauses derived from all state machine transitions, which are consistent with the state machine, are combined with any preexisting clauses to serve as the specification. Although other test criteria 1 could be applied [8, 14], we confine ourselves here to a specification mutation adequacy criterion. Simply stated, the criterion is that a test set must kill all mutations of a specification produced with some set of mutation operators. A mutant is killed if the mutant CTL clause is shown inconsistent with the trace, or ....

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In Proceedings of the Joint 7th European Software Engineering Conference and 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, Toulouse, France, September 1999.

....outline the TRIO system, which facilitates the semi automatic creation of test cases from temporal logic specifications. Chang et al. 95] describes a flexible, heuristic driven, programmable scheme for generating tests from specifications in ADL, a language based on first order predicate logic. Gargantini 99] illustrates how a model checker can be used to create tests from state machine specifications, and [Bauer Lamb 79] discusses how to derive test cases from functional requirements for state machines. These efforts all require a formal, modular and accurate specification of the system to be tested ....

Gargantini, A. and Heitmeyer, C. "Using Model Checking to Generate Tests from Requirements Specifications." In Proceedings of the Joint 7th Eur. Software Engineering Conference and 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering. Toulouse, France, September 1999.

No context found.

Gargantini, A. and C. Heitmeyer, Using model checking to generate tests from requirements specifications, Proceedings, ACM 7th European Software Engineering Conference/7th ACM SIGSOFT Symposium on Foundations of Software Engineering, Sept.

....example, if the animation goal a is WaterPres 500 , we have to find a sequence of values for monitored variables such that the system reaches a state where WaterPres becomes greater than 500. Note that the notion of animation goal is very similar to test goal or test predicate as presented in [6,9], as well animation sequence is similar to test sequence. For SCR specifications, we distinguish (as explained in Section 2) between requirements that refer to safety properties of the system, and functional requirements that are specified by tables and refer to the operation of the system. In ....

....state where such event occurs. Example 3. The 5 animation goals for event table 3 defining Overridden are: Pressure = High F(Pressure=High) T(Reset=On) T(Reset=On) 3. 2 Generation of Animation Sequences For automatic scenario generation, we use the method proposed in [6,9] which exploits the model checkers Spin [15] or SMV [20] and, in particular, their ability to generate counter examples. The method consists in the following steps. First, we encode the SCR specification in the language of the model checker (Spin or SMV) following the technique described in [1] ....

[Article contains additional citation context not shown here]

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In O. Nierstrasz and M. Lemoine, editors, Proceedings of the 7th European Engineering Conference and the 7th ACM SIGSOFT Symposium on the Foundations of Software Engineering, volume 1687 of LNCS, Sept. 6--10 1999.

....Weyuker 1993] A specification can also provide selection criteria to choose, among all possible system behaviors, an adequate test suite. Normally a selection criterion introduces some algorithms or techniques to actually generate test sequences from formal specifications [Blackburn et al. 1997, Gargantini and Heitmeyer 1999, Weyuker et al. 1994] In this paper, we focus on ASM based testing, and tackle several issues in this topic. In particular, we attempt to solve the challenging problem concerning the exploitation of ASMs for defining and implementing methods for generating test suites from high level ....

....Every rule models a particular behavior or decision of the specified software. In testing, we like to recreate that behavior or that decision determining, if possible, when the decision is taken. Moreover, we want to check the system behavior when the decision is not taken (else cases discussed in [Gargantini and Heitmeyer 1999]) Since each rule guard models the event or the state condition under which the decision is taken or the behavior is exposed, we search for a state where the guard is true (hence the rule fires and the decision is taken) as well for a state where the guard is false (hence the rule does not fire ....

[Article contains additional citation context not shown here]

Gargantini, A. and Heitmeyer, C. (1999). Using model checking to generate tests from requirements specifications. In Nierstrasz, O. and Lemoine, M., editors, Proceedings of the 7th European Engineering Conference and the 7th ACM SIGSOFT Symposium on the Foundations of Software Engeneering, volume 1687 of LNCS.

No context found.

Gargantini, A. and Heitmeyer, C. (1999). Using model checking to generate tests from requirements specifications. In Proceedings of the 7th European Software Engineering Conference, Held Jointly with the 7th ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE'99, volume 1687 of LNCS, pages 146--162. Springer-Verlag.

No context found.

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In Proceedings of the 7th European engineering conference held jointly with the 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE). Springer-Verlag, 1999.

No context found.

A. Gargantini and C. Heitmeyer, "Using Model Checking to Generate Tests from Requirements Specifications", Proc., ESEC'99 & 7th ACM SIGSOFT Intern. Symp. on Foundations of Software Eng. (ESEC/FSE99), Toulouse, September 1999.

No context found.

Gargantini, A., Heitmeyer, C.: Using model checking to generate tests from requirements specifications. In Nierstrasz, O., Lemoine, M., eds.: ESEC/FSE '99: Seventh European Software Engineering Conference and Seventh ACM SIGSOFT Symposium on the Foundations of Software Engineering. Volume 1687 of Lecture Notes in Computer Science., Toulouse, France, Springer-Verlag (1999) 146--162

No context found.

A. Gargantini and C. Heitmeyer, "Using Model Checking to Generate Tests from Requirements Specifications", Proc., ESEC'99 & 7th ACM SIGSOFT Intern. Symp. on Foundations of Software Eng. (ESEC/FSE99), Toulouse, September 1999.

No context found.

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. Software Engineering Notes, 24(6):146--162, November 1999.

No context found.

A. Gargantini and C. Heitmeyer. Using model checking to generate tests from requirements specifications. In Proc. 7th European Engineering Conference held jointly with the 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 146--162. Springer-Verlag, 1999.

No context found.

A. Gargantini and C. Heitmeyer, \Using model checking to generate tests from requirements specications", in Proc. Joint 7th European Software Eng. Conference and 7th ACM SIGSOFT Int. Symp. on Foundations of Software Eng., September 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC