NE Fenton, M Neil, "A Critique of Software Defect Prediction Models ", in IEEE Transactions on Software Engineering vol 25 no 5 (Sep/Oct 1999) pp 675--689; at http://www.dcs.qmul.ac.uk/~norman/papers/defects_ prediction_preprint105579.pdf  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... to describe the predictive relationship between software metrics and the classification of the software components into fault prone and non fault prone categories [6] These techniques include discriminant analysis [9] classification trees [10] pattern recognition [11] Bayesian network [12], case based reasoning (CBR) 13] and regression tree models [6] There are also some prototype or tools [16,17,18,19] that use such techniques to automate the procedure of software quality prediction. However, these tools address only one kind of metrics, e.g. process metrics or static code ....

.... In order to predict the quality of different software components, several techniques have been developed to classify software components according to their reliability [6] These techniques include discriminant analysis [9] classification trees [10] pattern recognition [11] Bayesian network [12], case based reasoning (CBR) 13] and regression tree model [6] In ComPARE, we integrate five types of models to evaluate the quality of the software components for an overall CBSD system evaluation. User can customize these models and compare the prediction results from different tailor made ....

[Article contains additional citation context not shown here]

N.E.Fenton and M.Neil, "A Critique of Software Defect Prediction Models," IEEE Transactions on Software Engineering, SE-25(5), pp.675-689, Oct. 1999. 15

....by a fault may lead to a whole network crash or to an inability to interpret an address with specific characters in it. These types of failures are not the same, the former being more serious. Lack of accounting of fault severity was one of the criticisms of the quality modeling literature in [49]. In general, unless the organization has a reliable data collection program in place where severity is assigned, it is difficult to retrospectively obtain this data. Therefore, the prediction models developed here can be used to identify classes that are prone to have faults that cause any type ....

N. Fenton and M. Neil: "A Critique of Software Defect Prediction Models". In IEEE Transactions on Software Engineering, 25(5):676-689, 1999.

....U shaped curve is depicted in Figure 1 as it is often presented in the literature. It shows that fault density is at a minimal value at a certain component size. Recently, Hatton [29] has articulated this theory forcefully and proposed a cognitive mechanism that would explain it. Fenton and Neil [20] term this theory the Goldilocks Conjecture . size fault density (faults size) A B Figure 1: U shaped curve relating fault density to size that exemplifies the optimal size theory. The curve can be broken up into two parts: A and B. The evidence that exists sometimes supports the whole curve, ....

....would start to rise again for larger components. Moller and Paulish [39] found that components less than 70 LOC tended to have a higher fault density than larger components. These results mean that reducing the size of components is detrimental to software quality. As noted by Fenton and Neil [20], this is counter to one The systems were implements in Pascal, PL S, and assembler respectively. 87 09 04 00 3 of the axioms of software engineering, namely program decomposition. In fact, this is one of the more explicit and controversial claims made by proponents of the optimal size theory ....

[Article contains additional citation context not shown here]

N. Fenton and M. Neil: "A Critique of Software Defect Prediction Models". In IEEE Transactions on Software Engineering, 25(5):676-689, 1999.

.... is data mining not used more in software engineering (SE) Recent reviews of the use of data mining in SE (e.g. 15, 16] report that data mining in SE is a mature technique based on widely available tools using well understood algorithms (e.g. neural nets or decision tree learners or Bayes nets [3, 8, 11, 24]) Also, impressive results have been reported in certain domains (e.g. cost estimation [3] or prediction of faulty modules [28] Further, data mining tools that can handle very large data sets now come bundled and integrated with standard commercial packages such as Microsoft s SQL Server 2000 ....

....5. Related Work Bayesian reasoning has been used to sketch out subjec400 tive knowledge (e.g. our software management oracle) then assess and tune that knowledge based on available data. Success with this method includes the COCOMO II effort estimation tool [3] and defect prediction modelling [8]. In domains that lack the data required for the tuning (e.g. our test domains) Bayesian reasoning is impractical. Treatment learners solve a problem that is subtlety different from the standard machine learning problem solved by (e.g. C4.5 [23] Standard learners assume that all classes are of ....

N. E. Fenton and M. Neil. A critique of software defect prediction models. Software Engineering, 25(5):675 689, 1999. Available from http://citeseer.nj.nec. corn/fent on 9 9 cr it ique. htrnl.

....by a fault may lead to a whole network crash or to an inability to interpret an address with specific characters in it. These types of failures are not the same, the former being more serious. Lack of accounting of fault severity was one of the criticisms of the quality modeling literature in [28]. In general, unless the organization has a reliable data collection program in place where severity is assigned, it is difficult to retrospectively obtain this data. Therefore, the prediction models developed here can be used to identify classes that are prone to have faults that cause any type ....

N. Fenton and M. Neil: "A Critique of Software Defect Prediction Models". To appear in IEEE Transactions on Software Engineering.

....by a fault may lead to a whole network crash or to an inability to interpret an address with specific characters in it. These types of failures are not the same, the former being more serious. Lack of accounting of fault severity was one of the criticisms of the quality modeling literature in [48]. In general, unless the organization has a reliable data collection program in place where severity is assigned, it is difficult to retrospectively obtain this data. Therefore, the prediction models developed here can be used to identify classes that are prone to have faults that cause any type ....

N. Fenton and M. Neil: "A Critique of Software Defect Prediction Models". To appear in IEEE Transactions on Software Engineering.

....USA. shesh.rai stjude.org To appear in the IEEE Transactions on Software Engineering. 2001. V02 27 02 00 2 at a minimal value at a certain component size. Recently, Hatton [25] has articulated this theory forcefully and proposed a cognitive mechanism that would explain it. Fenton and Neil [16] term this theory the Goldilocks Conjecture . size fault density (faults size) A B Figure 1: U shaped curve relating fault density to size that exemplifies the optimal size theory. The curve can be broken up into two parts: A and B. The evidence that exists sometimes supports the whole curve, ....

....would start to rise again for larger components. Moller and Paulish [34] found that components less than 70 LOC tended to have a higher fault density than larger components. These results mean that reducing the size of components is detrimental to software quality. As noted by Fenton and Neil [16], this is counter to one of the axioms of software engineering, namely program decomposition. In fact, this is one of the more explicit and controversial claims made by proponents of the optimal size theory [23] 24] 26] 6 The systems were implements in Pascal, PL S, and assembler respectively. ....

[Article contains additional citation context not shown here]

N. Fenton and M. Neil: "A Critique of Software Defect Prediction Models". In IEEE Transactions on Software Engineering, 25(5):676-689, 1999.

....Estimation of software fault proneness after a testing session can provide feedback on the testing activity and help in de ning delivery and maintenance procedures. Software fault proneness, de ned as the probability of presence of faults in the software, cannot be directly measured on software [FN99] However, fault proneness can be estimated based on software attributes directly measurable on software, if relations are found between these attributes and fault proneness. Research in this eld has been directed in two main directions: de nition of metrics to capture software complexity and ....

....aim at investigating how to nd speci c solutions based on the available domain knowledge. To this end, many methods have been explored, based on machine learning principles such as decision trees [SP88, PS90] or neural networks [KLP94] probabilistic approaches suchasBayesian Belief Networks [FN99] statistical techniques such as discriminant analysis [MK92] and regression [MR00] or mixed techniques such as optimized set reduction [BBT92] Some of the proposed methods give only a discrete characterization of potentially faulty modules, i.e. classify modules as fault prone or ....

[Article contains additional citation context not shown here]

Norman E. Fenton and Martin Neil. A critique of software defect prediction models. IEEE Transactions on Software Engineering, 25(5):675-689, September/October 1999.

....Use the concept of version space and the candidate elimination algorithm in CL to learn the definition of z. Software defect prediction Software defect prediction is a very useful and important tool to gauge the likely delivered quality and maintenance effort before software systems are deployed [4]. Predicting defects requires a holistic model rather than a single issue model that hinges on either size, or complexity, or testing metrics, or process quality data alone. It is argued in [4] that all these factors must be taken into consideration in order for the defect prediction to be ....

....tool to gauge the likely delivered quality and maintenance effort before software systems are deployed [4] Predicting defects requires a holistic model rather than a single issue model that hinges on either size, or complexity, or testing metrics, or process quality data alone. It is argued in [4] that all these factors must be taken into consideration in order for the defect prediction to be successful. Bayesian Belief Networks (BBN) prove to be a very useful approach to the software defect prediction problem. A BBN represents the joint probability distribution for a set of variables. ....

[Article contains additional citation context not shown here]

N. Fenton and M. Neil, "A critique of software defect prediction models," IEEE Trans. SE, Vol. 25, No. 5, Sept. 1999, pp. 675-689.

....[69] concluded that principal components are unstable across different products in different organizations. This suggests that the definition of a domain metric will be different across studies, making the accumulation of knowledge about product metrics rather tenuous. Furthermore, as noted in [40], such domain metrics are difficult to interpret and act upon by practitioners. In general, the use of principal components or domain metrics as variables in models is not advised. In addition to the above disadvantages, there are other ways of dealing with multicollinearity, therefore there is ....

N. Fenton and M. Neil, "A Critique of Software Defect Prediction Models," IEEE Transactions on Software Engineering, vol. 25, no. 5, pp. 676-689, 1999.

.... perspectives [Courtois et al. 1998, Delic at al 1997, Fenton et al. 1998, SERENE 1999] provide improved reliability predictions of prototype military vehicles (the TRACS project, TRACS 1999] predict general software quality attributes such as defect density and cost (the IMPRESS project [Fenton and Neil 1999, Lewis et al. 1998] Making Decisions BBNs and MCDA 4 April 2000 Page 3 of 27 In consultancy projects we have used BBNs to . assess safety of PES components in the railway industry; provide predictions of insurance risk and operational risk; predict defect density of software in ....

Fenton NE and Neil M, "A Critique of Software Defect Prediction Models", IEEE Transactions on Software Engineering, 25(3), 1999.

....other process and product measures such as the size of or the number of fixes in previous releases, sub system or module size, testing effort and so on. When enough metric data is available, and the process is sufficiently mature, models such as Bayesian nets may be useful to predict defect rates [fen99]. The above examples all relate to fault related aspects of quality. Other measures may be defined, collected and analysed in an analogous manner. In summary we observe that the underlying cause of the seventh law, the decline of software quality with age, appears to relate to a growth in ....

Fenton NE and Neil M, A Critique of Software Defect Prediction Models, 25(3) IEEE Trans. on Softw. Eng., 1999

....by a fault may lead to a whole network crash or to an inability to interpret an address with specific characters in it. These types of failures are not the same, the former being more serious. Lack of accounting of fault severity was one of the criticisms of the quality modeling literature in [32]. In general, unless the organization has a reliable data collection program in place where severity is assigned, it is difficult to retrospectively obtain this data. Therefore, the prediction models developed here can be used to identify classes that are prone to have faults that cause any type ....

N. Fenton and M. Neil: "A Critique of Software Defect Prediction Models". To appear in IEEE Transactions on Software Engineering.

....analysis, and hence do not yet address the primary objective of metrics. In the case of quality prediction (see for example, 7,24,39,46] the emphasis has been on determining regression based models of the form f(complexity metric) defect density where defect density is defects per KLOC. In [20] we provided an extensive critique of this approach. We concluded that the existing models are incapable of predicting defects accurately using size and complexity metrics alone. Furthermore, these models offer no coherent explanation of how defect introduction and detection variables affect ....

Fenton NE and Neil M, A Critique of Software Defect Prediction Models, IEEE Transactions on Software Engineering, to appear, 1999.

....In many cases it is the only driver, although recently resource and process quality factors have been considered. Solution problem size complexity Resources process quality Complexity Functionality Quality of staff, tools Number of defects Figure 1 Classical approach to defect modelling In [Fenton and Neil 1999] we provided an extensive critique of this classical approach. We identified problems such as: Fails to distinguish different notions of defect . Statistical methods are often flawed . Size is wrongly assumed to be a causal factors for defects . Obvious causal factors are not modelled . ....

.... perspectives [Courtois et al. 1998, Delic at al 1997, Fenton et al. 1998, SERENE 1999] provide improved reliability predictions of prototype military vehicles (the TRACS project, TRACS 1999] predict general software quality attributes such as defect density and cost (the IMPRESS project [Fenton and Neil 1999, Lewis et al. 1998] In consultancy projects we have used BBNs to . assess safety of PES components in the railway industry; provide predictions of insurance risk and operational risk; predict defect counts for software modules in consumer electronics products. The BBN in Figure 7 is a ....

Fenton NE and Neil M, A Critique of Software Defect Prediction Models, IEEE Transactions on Software Engineering, to appear, 1999.

.... from different industrial perspectives [Courtois et al. 1998, Delic at al 1997, Fenton et al. 1998, SERENE 1999] provide improved reliability predictions of prototype military vehicles (TRACS project) predict general software quality attributes such as defect density and cost (IMPRESS) [Fenton and Neil 1999, Lewis et al. 1998] In consultancy projects we have used BBNs to . assess safety of PES components in the railway industry . provide predictions of insurance risk . predict reliability of software in consumer electronics products. Making Decisions BBNs and MCDA 3 August 1999 Page 3 of 15 In ....

Fenton NE and Neil M, "A Critique of Software Defect Prediction Models", IEEE Transactions on Software Engineering, 25(3), 1999.

No context found.

NE Fenton, M Neil, "A Critique of Software Defect Prediction Models ", in IEEE Transactions on Software Engineering vol 25 no 5 (Sep/Oct 1999) pp 675--689; at http://www.dcs.qmul.ac.uk/~norman/papers/defects_ prediction_preprint105579.pdf

No context found.

Fenton, Norman E. A Critique of Software Defect Prediction Models. IEEE Transactions on Software Engineering, Vol. 25, No 3, May/June 1999.

No context found.

N. E. Fenton and M. Neil, "A Critique of Software Defect Prediction Models", IEEE Transactions on Software Engineering, 25, pp. 675-689, 1999.

No context found.

Fenton, N. E., & Neil, M. (1999). A critique of software defect prediction models. IEEE Transactions on Software Engineering, 25(5), 675-689.

No context found.

N. E. Fenton and M. Neil, "A critique of software defect prediction models", IEEE Transactions on Software Engineering 25(5) (1999) 675--689.

No context found.

N. Fenton and M. Neil: "A Critique of Software Defect Prediction Models". In IEEE Transactions on Software Engineering, 25(5):676-689, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC