E. Kamsties and C.M. Lott. An empirical evaluation of three defect-detection techniques. In Proceedings of the Fifth European Software Engineering Conference, pages 362--383, 1996.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....Hence, although, desirable this aspiration may not be feasible in the modern world of urgent demands. Having said this, a few brave software engineering researchers have attempted to produce replication packages to increase the reproducibility of their work. The package by Kamatics and Lott [17] represents an excellent example of such a package. Although the package provides much useful information to allow a replication, it is difficult to see how such a package could ever be complete in the software engineering context. Although guidelines themselves are insufficient to ensure high ....

E Kamsties, C M Lott, "An empirical evaluation of three defect detection techniques", International Software Engineering Network Technical Report, ISERN-95-02, 1995.

....is based on work performed earlier by Hetzel [53] The subjects were 59 highly experienced data processing professionals testing and inspecting a PL I program. Myers [95] reports an average effectiveness value of 38 for inspections. This controlled experiment was replicated several times [7] [66] [139] with similar results. A. Gately [46] presents some results from the Raytheon Systems Company. In this study, historical review data from a large real time embedded system were analyzed. She found an average number of defects of 18.2 defects. Conradi et al. 27] present some review results ....

Kamsties, E. and Lott, C. M., 1995. An empirical evaluation of three defectdetection techniques. In: Schfer, W. and Botella, P., editors, Proceedings of the 5th European Software Engineering Conference, pages 362-383. Lecture Notes in Computer Science Nr. 989, Springer-Verlag.

.... analysis (Kitchenham, 1996; Kitchenham, 1996 1997) A method for developing SW quantitative case studies (Kitchenham, 1996) Moreover, many papers reported about lessons learned while running new or replicated experiments (Basili, et al. 1996; Briand, et al. 1996; Ciolkowsky, et al. 1997; Kamstie and Lott, 1995; Porter, et al. 1995; Visaggio, 1997; and see the ISERN web page for further references) Finally, a Process Improvement Experiment project (PIE) was launched in Europe, to develop a standard like TT meta model (see the ESI web page) Our work reflects our having taken the following positions. ....

Kamstie E, Lott CM. 1995. An empirical evaluation of three defect-detection techniques., ISERN Technical Report 95-02. Fraunhofer Institute for Experimental Software Engineering, University of Kaiserlautern: Kaiserlautern, Germany.

....earlier by Hetzel [Hetzel, 1976] The subjects were 59 highly experienced data processing professionals testing and inspecting a PL I program. Myers reports an average effectiveness value of 38 for inspections. This controlled experiment was replicated several times [Basili and Selby, 1987] [Kamsties and Lott, 1995], Myers, 1978] Wood et al. 1997] with similar results. 4.2.2 Cost It is necessary for a project manager to have a precise understanding of the cost associated with inspections. Since inspection is a human based activity, inspection costs are determined by human effort. The most important ....

Kamsties, E. and Lott, C. M. (1995). An empirical evaluation of three defect-detection techniques. In Proceedings of the Fifth European Software Engineering Conference, pages 362--383. Lecture Notes in Computer Science Nr. 989, Springer-Verlag.

....to some predetermined criteria, such as branch coverage or variable definition and use testing. Tests are designed to analyse the flow of control and data within code. In the last 25 years much research has been undertaken on the advantages and disadvantages of these techniques. It is considered [KL95] that functional testing is cheaper in terms of faster test data generation and defect detection. Functional testing can locate faults of both omission and commission such as missing features and incorrect execution paths. Functional testing techniques tend to generate large quantities of data ....

.... strategies generally suffer from analysis of either small programs or a small 2 number of test subjects (testers) Most suggest that functional testing is a useful preliminary to code testing, creating a base of useful test cases with generally high code coverage [POC93, BS87, Mye78, HH79, KL95, HLL94] It is agreed that tests performed later in the software development process are more expensive to correct [Bei90, Mar91] and hence functional testing coupled with heavy unit testing can be a cheap and effective strategy. Horgan et al. HLL94] also recommend that both functional and ....

[Article contains additional citation context not shown here]

E. Kamsties and C.M. Lott. An Empirical Evaluation of Three Defect-Detection Techniques. In ESEC, 1995.

.... ISERN Report ISERN 98 10 the form of controlled experiments compared the (cost )effectiveness of inspection and testing when applied independently of each other, i.e. the subjects of these experiments applied either inspection or testing on a software artifact but not both [3] 12] [17], 21] 28] However, as pointed out in [28] there is little clear, consistent evidence that one defect detection technique is more (cost )effective than another when applied in isolation. The existing evidence to date rather suggests that each defect detection technique has its own merits, ....

....in isolation, Selby also looked at all six possible pairwise combinations of the three techniques across hypothetical teams of two. He found that, on average, the combinations detected 67.5 of the defects in comparison with an average of 49.8 of individual techniques. Kamsties and Lott [17] have replicated the Basili Selby experiment twice. They found no statistically significant difference regarding the effectiveness of the three techniques in either of the replications. However, they did find that functional testing is the most cost effective technique. Most recently, Wood et al. ....

[Article contains additional citation context not shown here]

E. Kamsties and C. M. Lott. An empirical evaluation of three defect-detection techniques. In W. Schäfer and P. Botella, editors, Proceedings of the Fifth European Software Engineering Conference, pages 362--383. Lecture Notes in Computer Science Nr. 989, Springer -Verlag, Sept. 1995.

....familiar with, instead of finding defects. Since the subjects were competent in C , material in that language was chosen for the experiment. The decision was further ratified by the availability of high quality material from a local replication of Kamsties and Lott s defect detection experiment [16]. For the training materials, a selection of programs originally used in Kamsties and Lott s experiment were used, since each program had an appropriate specification, a list of library functions used by the program and a comprehensive fault list. These programs were originally written in non ANSI ....

....number of participants. For example, the process used did not involve the author presenting an overview of the product, and a rework phase was not used. However, the detection collection approach to inspection is a standard process [12] These threats are typical of many empirical studies, e.g. [25, 16]. These threats can be reduced by performing repeated experimentation with other subjects, programs and processes. Program analyse.cc graph.cc Section 1 2 1 2 Method Tool Paper Paper Tool Groups 7 7 7 7 Mean 10.86 10.71 9.57 8.86 St. Dev. 0.69 0.95 1.27 1.07 St. Error 0.26 0.36 0.48 0.40 F ....

E. Kamsties and C. M. Lott. An empirical evaluation of three defect-detection techniques. Technical Report ISERN-95-02, International Software Engineering Research Network, May 1995.

....familiar with, instead of finding defects. Since the subjects were competent in C , material in that language was chosen for the experiment. The decision was further ratified by the availability of high quality material from a local replication of Kamsties and Lott s defect detection experiment [17]. For the training materials, a selection of programs originally used in Kamsties and Lott s experiment were used, since each program had an appropriate specification, a list of library functions used by the program and a comprehensive defect list. These programs were originally written in ....

....used did not involve the author presenting an overview of the product, giving no context for the inspection. A rework phase was also not used. However, the detection collection approach used in the experiment is a standard process [12] These threats are typical of many empirical studies, e.g. [17, 25]. They can be reduced by internal and external replication of the experiment, with other subjects, programs and processes. 3 ASSIST Version 1 The first version of ASSIST provided the basic facilities required of any inspection support tool, along with several innovations. These innovations ....

E. Kamsties and C. M. Lott. An empirical evaluation of three defect-detection techniques. Technical Report ISERN-95-02, International Software Engineering Research Network, May 1995.

....statement coverage as its stopping criteria and code reading by stepwise abstraction. They reported weak evidence that code reading was more effective. Again they suggested that a combinational approach of defect finding strategies seemed to offer advantages. ffl More recently Kamsties and Lott[24] replicated the Basili and Selby experiment twice. They reported no significant differences between the three techniques. ffl Finally, Roper et al. 35] replicated the Kamsties and Lott experiment. Again, they reported no significant differences, but again, reported that the different techniques ....

E. Kamsties and C.M. Lott. An empirical evaluation of three defectdetection techniques. In Proceedings of the Fifth European Software Engineering Conference, pages 362--383, 1996.

....ffl In the study undertaken by Hetzel, the structured testing component has been replaced by a functional and structural testing component, hence we have given this study a weighting of zero, i.e. excluded it from the meta analysis. ffl Kamsties and Lott state in their technical report[23] that the design of the initial experiment (but not the subsequent replication) was flawed . Hence, again, we have given the initial study a weighting of zero, but we have given the subsequent replication a full weighting of one. Other points could have been considered (such as the limited ....

E. Kamsties and C.M. Lott. An empirical evaluation of three defect detection techniques. Technical report, ISERN-95-02, International Software Engineering Network, 1995.

....subjects was that they valued the experience and learned much. We have made all materials available for software engineering empiricists who may be interested in replicating or further extending the K L experiment, or those who might like to use the experiment as a teaching tool. The materials in [KL95] include the instruments, lists of faults, data collection forms, and a description of all processes involved in running the experiment, extracting the data, and performing the analyses. We are interested in learning whether external replications confirm our results that individual differences are ....

Erik Kamsties and Christopher M.Lott. An empirical evaluation of three defect-detection techniques. Technical Report ISERN 95--02, Department of Computer Science, University of Kaiserslautern, 67653 Kaiserslautern, Germany, May 1995.

....subjects was that they valued the experience and learned much. We have made all materials available for software engineering empiricists who may be interested in replicating or further extending the K L experiment, or those who might like to use the experiment as a teaching tool. The materials in [KL95] include the instruments, lists of faults, data collection forms, and a description of all processes involved in running the experiment, extracting the data, and performing the analyses. We are interested in learning whether external replications confirm our results that individual differences are ....

Erik Kamsties and Christopher M. Lott. An empirical evaluation of three defect-detection techniques. Technical Report ISERN 95--02, Department of Computer Science, University of Kaiserslautern, 67653 Kaiserslautern, Germany, May 1995.

....for syntactical and semantical correctness. Program tokens sorts and counts words in its input. Static source code measures of the programs are summarized in Table 3, and reveal a size difference in excess of 2:1. Previous experience with these exercises in the context of a controlled experiment [Kamsties and Lott, 1995] indicated that the structural testing exercise required more time than the functional testing exercise for the same program, so the size difference was necessary to keep the exercise duration as similar as possible. Total Blank Lines w NB, NC Semi Fault Program lines lines comments lines colons ....

....All faults caused unique, visible failures given suitable inputs. 4.3. 3 Data used to derive quality models The data supplied as quantitative quality models (e.g. estimated amount of time required) were derived from previous experimental trials that used the same techniques and instruments [Kamsties and Lott, 1995]. Table 4 summarizes the previous observations and the quality models that were supplied to all subjects. The guidance materials explained that the quantitative quality models were based on similar components and exercises. With respect to the quality model for failure count, in a preset ....

Kamsties, E. and Lott, C. M. (1995). An empirical evaluation of three defect-detection techniques. In Schafer, W. and Botella, P., editors, Proceedings of the Fifth European Software Engineering Conference, pages 362--383. Lecture Notes in Computer Science Nr. 989, Springer--Verlag.

....Process [10] We hope that our efforts will make it possible for the experiment to become a standard exercise that developers will use to evaluate and sharpen their defectdetection skills. The paper provides an overview of two internal replications of the extended experiment (the K L experiment ) [13]. We present our hypotheses, the experimental design, the defect detection techniques (treatments) the programs and faults (instruments) and the experimental procedure (including the analysis approach and choice of dependent variables) Then we give results from the two replications and compare ....

....subjects was that they valued the experience and learned much. We have made all materials available for softwareengineering empiricists who may be interested in replicating or further extending the K L experiment, or those who might like to use the experiment as a teaching tool. The materials in [13] include the instruments, lists of faults, data collection forms, and a description of all processes involved in running the experiment, extracting the data, and performing the analyses. We are interested in learning whether external replications confirm our results that individual differences are ....

Erik Kamsties and Christopher M. Lott. An empirical evaluation of three defect-detection techniques. Technical Report ISERN 95--02, Department of Computer Science, University of Kaiserslautern, 67653 Kaiserslautern, Germany, May 1995.

No context found.

E. Kamsties and C.M. Lott. An empirical evaluation of three defect-detection techniques. In Proceedings of the Fifth European Software Engineering Conference, pages 362--383, 1996.

No context found.

Erik Kamsties and Christopher Lott. An empirical evaluation of three defect detection techniques. Technical Report ISERN 95-02, Dept. Computer Science, Universityof Kaiserslautern, May1995.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC