J. Voas, L. Morell, and K. Miller. Predicting where faults can hide from testing. IEEE Software, 8:41--48, March 1991.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....error handlers must bring the system back to a safe state. 1) Failure Tolerance [41] TESTABILITY Only testable requirements should be specified. Code should be designed to be testable. 1) Domain Range Ratio (DRR) 40] 2) Controllability [2] 3) Observability [2] 4) Sensitivity Analysis [39] USABILITY Requirements should be formulated with 1) Total Training Time [38] NFRs should be testable, and quantitative so that metrics can be easily applied and measured. For example, a poor non functional requirement (NFR) is one that cannot be easily tested and is un quantifiable, e.g. An ....

VOAS, J. M.; MORELL, L.; MILLER, K. W.: "Predicting Where Faults Can Hide From Testing" IEEE Software, 1991, 8 (2) pp. 41 - 48.

....because all thread data needs to be stored. Test templates are sets of conditions for each test case that confirm the test case satisfy the selected covering criteria and are more abstract then test case. Test templates allow: 1. to obtain test cases for instance, by a random test data generator [6] and 2. to check existing test cases for conformity with covering criteria. Our simplified ATPG algorithm is presented on figure 2. and requires an UML model extended only by a small set of controlling features. Figure 1.b. shows how the Bubble sort algorithm may be modeled using the extended ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2), 41-48, March 1991

....with high level of testability. The potential benefits of measuring software testability are significant: testing resources can be distributed more effectively, and the degree to which software component s testing is to be performed could be estimated using the component s testability measure [10]. Voas and Miller [9] propose static testability measurement approach, called Propagation Infection Execution. This method is based on input output domains in random black box testing. Le Traon and Robach [8] propose designspecification phase testability measurement in terms of ....

J. Voas, K., L. Morell, K. Miller, "Predicting Where Faults Can Hide from Testing", IEEE Software (March 1991), pp.41-48.

....such as mutation hold great promise for improving the quality of software. Although most of these techniques are currently so expensive that practical use is rare, recent advances in mutation research has brought practical mutation testing system closer to reality. Commercial tools (e.g. PiSCES [VMM91] have recently become available, although they are still not widely used in industry. A major problem with mutation testing is that it is too expensive to apply; one aspect of the cost is identifying programs that are intended to be faulty, but in fact are not. These programs are said to be ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2):41--58, March 1991.

.... kinds of test data generators: path wise test data generators to cover certain structural elements in the program [Kor90, DO91, OJP99, Cla76, BKM91, How77, RbFHC76, BBS 79] data specification generators to generate test data from a formal grammar [Mau90] and random test data generators [VMM91] Most of the path wise test data generators [Cla76, BKM91, How77, RbFHC76, BBS 79] have used symbolic execution and they do not consider languages with exception mechanisms nor rigorously prove their soundness. Given a program and a designated path, the analysis symbolically executes the ....

Jeffrey Voas, Larry Morell, and Keith Miller. Predicting where faults can hide from testing. IEEE Software, 8(2):41--58, March 1991.

....Testability is a software metric that quantifies how difficult it is to test software. This level of difficulty could include the cost to generate test cases, write drivers or stubs, or determine correctness for a specific test case. The PIE assessment method for measuring testability [17] is based on the following testability definition: the probability that faults will result in observable failures for a given input distribution or test scheme. PIE implements this definition by computing three measurements: execution, which estimates the probability that a faulty statement will ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2), March 1991.

....attempt to nd a program input that will satisfy the testing requirement. The automation of test data generation is an important step in reducing the cost of software development and maintenance. A number of test data generation techniques have been automated. Random test data generators (e.g. [18, 20, 22]) select random inputs for the test data from some distribution. Structural or path oriented testdata generators typically use the program s control ow graph, select a particular path, and use a technique such as symbolic evaluation to generate test data for that path (e.g. 1, 3, 6, 15, 19] ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2), 41-48, March 1991. 19

....a set of mutant operators for C, which we henceforth call the Purdue C operators. To date, there are two publicized systems that use these operators. Delamaro et al. developed Proteum [8, 7] and Untch implemented TUMS [35, 36] to implement the schema based execution model [36] The Pisces system [37] also uses mutation to assess testability and to perform weak mutation [23] The Pisces system is partially based on the Purdue operators, and it is, to date, the only commercial tool that is based on mutation. O#utt et al. 32] defined a set of mutant operators for the Ada language. These ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2), March 1991.

....since coverage often points to it. 2) It may be hard to determine correctness of an interior routine from correctness of the system. For example, suppose the routine produces a complicated linked list structure and the system produces 0 or 1. Information is lost in that transformation ([Voas91a], Voas91b] A correct and incorrect linked list structure may both produce 1, so you can t test the interior routine via the system. This difficulty is usually because systems are not designed to be testable they are not designed to be easy to force into known states, and they do not provide ....

Jeffrey Voas, Larry Morell, and Keith Miller. "Predicting Where Faults Can Hide from Testing". IEEE Software, March 1991.

....Testability is a software metric that quantifies how difficult it is to test software. This level of difficulty could include the cost to generate test cases, write drivers or stubs, or determine correctness for a specific test case. The PIE assessment method for measuring testability [VMM91] is based on the following testability definition: the probability that faults will result in observable failures for a given input distribution or test scheme. PIE implements this definition by computing three measurements: execution, which estimates the probability that a faulty statement will ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2), March 1991.

....is a tool that helps the tester create test data. Test data generators have been categorized into three groups: structural oriented test data generators [BKM91, BEL75, Cla76, DO91, How77, Kor90, RHC76] data specification generators [BF79, MM75, Mau90] and random test data generators [Stu85, VMM91, MDL87] This paper focuses on structural oriented generators, which are based on covering certain structural elements in the program. Usually, these generators attempt to generate test data to meet a testing criterion such as path coverage, branch coverage, mutation, etc. Structural oriented ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2), March 1991. 27

.... into three groups: structural oriented test data generators attempt to cover certain structural elements in the program [5, 6, 7, 8, 9, 10, 11, 12] data specification generators generate test data from a formal description of the input domain [13, 14, 15] and random test data generators [16, 17, 18] create test data according to some distribution of the inputs without satisfying any test criterion. Because the analysis of the software is so complex, structural test data generators are intended to be used during unit testing. The ideas that have been presented in the past have not come into ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2):41--58, March 1991.

....of performance of monitoring by delegating the monitoring activities to a second processor (they call it a shadow processor) Their approach is very efficient; the monitored is nearly not slowed down. However, the set of monitoring commands they propose cannot be extended as in our approach. In [18], Voas needs to instrument source code to be able to detect hidden faults that cannot possibly be detected by simply comparing the output of the program with its expected output (coincidental correctness) Ball [1] instruments control flow graphs to implement non regression test case selection ....

J. Voas, L. Morell, and K. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2):41--48, March 1991.

....a set of mutant operators for C, which we henceforth call the Purdue C operators. To date, there are two publicized systems that use these operators. Delamaro et al. developed Proteum [8, 7] and Untch implemented TUMS [35, 36] to implement the schema based execution model [36] The Pisces system [37] also uses mutation to assess testability and to perform weak mutation [23] The Pisces system is partially based on the Purdue operators, and it is, to date, the only commercial tool that is based on mutation. Offutt et al. 32] defined a set of mutant operators for the Ada language. These ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2), March 1991.

....important to consider the test difficulties at an earlier step in the software life cycle. This is known as the Design for testability (DFT) In previous works, different DFT approaches have been proposed. They followed two main orientations: 1) the definition and evaluation of module testability [FREE91, VMMI91, PDKO93] and (2) the use of the testability criteria to transform a specification in order to obtain a more testable one [YPDK95] These two orientations are based on the evaluation of the testability. Evaluation techniques, such as the length of the test suite [PDKO93] are not always possible to apply, ....

....CC TCONconf TDATAreq DT TDATAind e e e TDISreq DR TDISind . e e e User A User B User A User B Figure 4. Relational representation of the User A view of the service. 4. TESTABILITY OF MODULES SPECIFIED BY RELATIONS Several works have presented a definition of the software testability [KDCK94, FREE91, VMMI91, YPDK95 and PDKO93]. Those definitions have many similarities, and most of them agree that a system (module, entity, function, program. is testable if it is controllable and observable. In this work, we focus on the definition given by Freedman [FREE91] The author affirm that a module is observable if it gives ....

J. Voas, L. Morrell and K. Miller, Predicting Where Faults Can Hide From Testing, IEEE Software, March 1991.

....pointers are pervasive and untyped, and a pointer can be derived for any variable by dereferencing . Additionally, C allows arithmetic operations to be applied to pointers, and even allows pointers to be mixed in expressions with integers and other types. Thus, C mutation systems such as PiSCES [VMM91] must deal with many complexities because of the language. In Ada, pointers are typed, and a limited number of operators can be applied to pointers. Thus, we are able to, in most cases, treat pointer types as just another kind of operand. A pointer reference is treated just as a variable ....

J. M. Voas, L. Morell, and K. W. Miller. Predicting where faults can hide from testing. IEEE Software, 8(2), March 1991.

....resulting in a lower software testability. Software sensitivity analysis is a code based technique based on the semantic definition of testability; it injects instrumentation that contains program mutation, data state mutation, and repeated executions to predict a minimum non zero fault size [7, 13]. The minimum non zero fault size is the smallest probability of failure likely to be induced by a programming error based upon the results of the injected instrumentation. Sensitivity analysis is not a testing technique, and thus it does not use an oracle, and can be completely automated ....

J. VOAS, L. MORELL, AND K. MILLER. Predicting Where Faults Can Hide From Testing. IEEE Software, 8(2):41--48, March 1991.

....testing strategy will force failures to be observable if faults were to exist and the probability that the oracle will be precise enough to detect failures. In this paper, we use an implementation of Voas s testability definition [12] to identify where faults (if they exist) are likely to hide [12, 9, 6]. Low testability source locations are statements where it is believed that small faults will be difficult to detect during testing. We thwart the potential error masking at low testability locations through strategic assertion placement; this will serve to improve the efficiency of fault ....

....The task of instrumenting the code with correct assertions at each location is burdensome, and there is no guarantee that the assertions themselves will be correct. Let L represent the set of all locations (statements) in a program, P , and suppose that propagation and infection analyses [12, 6] have been performed at each member of L. Let Theta l be the set of non zero infection estimates and the propagation estimate for the variable on the left hand side of some location, l; 5 Theta l does not contain l s estimate of the likelihood of reaching l. Now let l = min[ Theta l ] The ....

[Article contains additional citation context not shown here]

J. VOAS, L. MORELL, AND K. MILLER. Predicting Where Faults Can Hide From Testing. IEEE Software, 8(2):41--48, March 1991.

No context found.

J. Voas, L. Morell, and K. Miller. Predicting where faults can hide from testing. IEEE Software, 8:41--48, March 1991.

No context found.

J. Voas, L. Morell, and K. Miller. Predicting where faults can hide from testing. IEEE Software, 8:41--48, March 1991.

No context found.

J. Voas, L. Morrel and K. Miller, `Predicting where faults can hide from testing', IEEE Software, 8, (2), 41--48 (1991). Tutorial: Software Testing & Validation Techniques, IEEE Computer Society Press, 1981. Proceedings, 5th International Conference on Software Engineering, 1981.

No context found.

VOAS, J. M.; MORELL, L., and MILLER, K. W.: "Predicting Where Faults Can Hide From Testing" IEEE Software, 1991, 8 (2) pp. 41 - 48

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC