DMCA
Benchmarking Classification Models for Software Defect Prediction: A Proposed Framework and Novel Findings
| Venue: | IEEE Transactions On Software Engineering |
| Citations: | 86 - 1 self |
BibTeX
@ARTICLE{Lessmann_benchmarkingclassification,
author = {Stefan Lessmann and Student Member and Bart Baesens and Christophe Mues and Swantje Pietsch},
title = {Benchmarking Classification Models for Software Defect Prediction: A Proposed Framework and Novel Findings},
journal = {IEEE Transactions On Software Engineering},
year = {},
pages = {485--496}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract—Software defect prediction strives to improve software quality and testing efficiency by constructing predictive classification models from code attributes to enable a timely identification of fault-prone modules. Several classification models have been evaluated for this task. However, due to inconsistent findings regarding the superiority of one classifier over another and the usefulness of metric-based classification in general, more research is needed to improve convergence across studies and further advance confidence in experimental results. We consider three potential sources for bias: comparing classifiers over one or a small number of proprietary data sets, relying on accuracy indicators that are conceptually inappropriate for software defect prediction and cross-study comparisons, and, finally, limited use of statistical testing procedures to secure empirical findings. To remedy these problems, a framework for comparative software defect prediction experiments is proposed and applied in a large-scale empirical comparison of 22 classifiers over 10 public domain data sets from the NASA Metrics Data repository. Overall, an appealing degree of predictive accuracy is observed, which supports the view that metric-based classification is useful. However, our results indicate that the importance of the particular classification algorithm may be less than previously assumed since no significant performance differences could be detected among the top 17 classifiers. Index Terms—Complexity measures, data mining, formal methods, statistical methods, software defect prediction. Ç 1
Keyphrases
software defect prediction novel finding classification model metric-based classification statistical method proprietary data set advance confidence statistical testing procedure cross-study comparison accuracy indicator experimental result large-scale empirical comparison timely identification public domain data set several classification model predictive classification model formal method particular classification algorithm data mining code attribute software quality limited use significant performance difference potential source index term complexity measure comparing classifier nasa metric data repository predictive accuracy abstract software defect prediction strives inconsistent finding fault-prone module small number empirical finding comparative software defect prediction experiment
