P. Frankl, D. Hamlet, B. Littlewood and L. Strigini, "Choosing a Testing Method to Deliver Reliability", in 19th International Conference on Software Engineering (ICSE'97), pp.68-78, 1997.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....engineering process, we believe that one of the main concern is to provide a unified semantic between the analysis and the design models. Another approach to address software dependability is to provide mechanisms to improve reliability of software after it has been implemented. Works such as [5] use testing techniques to identify faults in the software that are likely to cause failures. Although they carry out an important research agenda, we believe that is cheaper to design and evaluate dependability concerns in the early stages of software engineering processes . Besides, levels of ....

P. Frankl, R. Hamlet, B. Littlewood, and L. Strigini. Choosing a Testing Method to Deliver Reliability. In International Conference on Software Engineering, pages 68--78, 1997.

.... test (using special values testing [2] ffl the aim of operational testing is to increase the reliability of the program under test (using tests that mimics the operational distribution of typical usage [2] It is only very recently that theoreticians have clearly separated the two practices [28]. In their article, Choosing a Testing Method to Deliver Reliability [28] Frankl, Hamlet, Littlewood and Strigini have clearly shown that the two testing practices above are distinct. In particular, they have highlighted the main shortcoming of defect testing: informally, defect testing may ....

.... is to increase the reliability of the program under test (using tests that mimics the operational distribution of typical usage [2] It is only very recently that theoreticians have clearly separated the two practices [28] In their article, Choosing a Testing Method to Deliver Reliability [28], Frankl, Hamlet, Littlewood and Strigini have clearly shown that the two testing practices above are distinct. In particular, they have highlighted the main shortcoming of defect testing: informally, defect testing may not lead to reliable software if the inputs that precipitate any of the ....

[Article contains additional citation context not shown here]

P. Frankl, D. Hamlet, B. Littlewood, and L. Strigini, "Choosing a testing method to deliver reliability," in Proceedings of the 19th International Conference on Software Engineering, pp. 68--78, ACM Press, May 1997.

....the adequacy of the testing that is carried out. Finally, the model can be used to study the relationship between component based test adequacy and the reliability of component based software. A good starting point for such a study would be the reliability framework described by Frankl et al. [5]. ACKNOWLEDGMENTS This material is based upon work supported by the National Science Foundation under Grant No. CCR 9701973. This effort was also sponsored by the Defense Advanced Research Projects Agency, and Rome Laboratory, Air 9 Force Materiel Command, USAF, under agreement number ....

P. Frankl, D. Hamlet, B. Littlewood, and L. Strigini, "Choosing a Testing Method to Deliver Reliability", Proc. 19th International Conference on Software Engineering, Boston, MA, pp. 68--78, 1997.

.... Earlier work [9] had yielded inconclusive results; however, the programs used in [9] were substantially smaller than [13, 20] On the analytic front, rigorous probabilistic models of the relationship between increasing whitebox coverage and the likelihood of fault detection have been developed [8, 11]. However, no known testing process is perfect; all known methods, white box or black box will let some faults slip The best current experimental work [13, 20] suggests that high levels of white box test coverage can guarantee high levels of fault detection. However, since white box testing is ....

P.G. Frankl, R. Hamlet, B. Littlewood, and L. Strigini. Choosing a testing method to deliver reliability. In Proceedings of the 19th International Conference on Software Engineering (To Appear). IEEE Computer Society, 1997.

....heavily to unreliability, so that the correction effort is cost effective. Several authors have published comparisons between classes of testing methods [1 9] Experimental comparisons have been inconclusive so far. Among comparisons based on modelling, we think the most advanced so far is in [Frankl et al. 1997, Frankl et al. 1998] to which we refer the reader for more extensive background material and whose analysis we seek to extend. Most previous studies of the effectiveness of testing methods used the probability of causing a failure, and thus finding a defect, as a measure of the effectiveness of ....

....of the effectiveness of a test series. This seems inappropriate when considering testing as a means for improving the software: what really matters is the reliability of the delivered software, hence the improvement that is obtained by applying the given testing method. Frankl and co authors [Frankl et al. 1997, Frankl et al. 1998] and, previously, Li Malaiya 1994] instead adopted as a measure of test effectiveness the increment in reliability that would be obtained with a given number of tests. Papers [Frankl et al. 1997, Frankl et al. 1998] show that the choice between testing methods depends on ....

[Article contains additional citation context not shown here]

P. Frankl, D. Hamlet, B. Littlewood and L. Strigini, "Choosing a Testing Method to Deliver Reliability", in 19th International Conference on Software Engineering (ICSE'97), pp.68-78, 1997.

....heavily to unreliability, so that the correction effort is cost effective. Several authors have published comparisons between classes of testing methods [1 9] Experimental comparisons have been inconclusive so far. Among comparisons based on modelling, we think the most advanced so far is in [8, 9], to which we refer the reader for more extensive background material and whose analysis we seek to extend. Most previous studies of the effectiveness of testing methods used the probability of causing a failure, and thus finding a defect, as a measure of the effectiveness of a test series. This ....

....of the effectiveness of a test series. This seems inappropriate when considering testing as a means for improving the software: what really matters is the reliability of the delivered software, hence the improvement that is obtained by applying the given testing method. Frankl and co authors [8, 9] (and, previously, 10] instead adopted as a measure of test effectiveness the increment in reliability that would be obtained with a given number of tests. Papers [8, 9] show that the choice between testing methods depends on rather subtle details of the assumed scenarios; and that debug 2 ....

[Article contains additional citation context not shown here]

P. Frankl, D. Hamlet, B. Littlewood and L. Strigini, "Choosing a Testing Method to Deliver Reliability", in Proc. 19th International Conference on Software Engineering (ICSE'97), 1997, pp. 68-78.

....these testing techniques are. There are several directions for continuing work, including further experimentation using other programs from the suite, further experimentation using different large programs, experiments using other measures of effectiveness, such as the failure rate after debugging [1] and experiments incorporating some means of measuring the cost of using a coverage criterion. Acknowledgments Bob Horgan s group at Bellcore developed ATAC and allowed us to use it. Alberto Pasquini and Paolo Matrella provided access to the the subject programs and provided the test generator. ....

P. G. Frankl, D. Hamlet, B. Littlewood, and L. Strigini. Choosing a testing method to deliver reliability. In Proceedings International Conference on Software Engineering, pages 68--78. IEEE Computer Society Press, May 1997.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC