DMCA
A.: Sound empirical evidence in software testing (2012)
| Venue: | In: ACM/IEEE International Conference on Software Engineering (ICSE |
| Citations: | 25 - 17 self |
BibTeX
@INPROCEEDINGS{Fraser12a.:sound,
author = {Gordon Fraser and Andrea Arcuri},
title = {A.: Sound empirical evidence in software testing},
booktitle = {In: ACM/IEEE International Conference on Software Engineering (ICSE},
year = {2012},
pages = {178--188}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract—Several promising techniques have been proposed to automate different tasks in software testing, such as test data generation for object-oriented software. However, reported studies in the literature only show the feasibility of the proposed techniques, because the choice of the employed artifacts in the case studies (e.g., software applications) is usually done in a non-systematic way. The chosen case study might be biased, and so it might not be a valid representative of the addressed type of software (e.g., internet applications and embedded systems). The common trend seems to be to accept this fact and get over it by simply discussing it in a threats to validity section. In this paper, we evaluate search-based software testing (in particular the EVOSUITE tool) when applied to test data generation for open source projects. To achieve sound empirical results, we randomly selected 100 Java projects from SourceForge, which is the most popular open source repository (more than 300,000 projects with more than two million registered users). The resulting case study not only is very large (8784 public classes for a total of 291,639 bytecode level branches), but more importantly it is statistically sound and representative for open source projects. Results show that while high coverage on commonly used types of classes is achievable, in practice environmental dependencies prohibit such high coverage, which clearly points out essential future research directions. To support this future research, our SF100 case study can serve as a much needed benchmark for test generation. Keywords-test case generation; unit testing; search-based software engineering; benchmark I.
Keyphrases
sound empirical evidence high coverage case study open source project internet application employed artifact non-systematic way test generation validity section sound empirical result popular open source repository object-oriented software sf100 case study chosen case study search-based software engineering search-based software different task practice environmental dependency valid representative software application abstract several evosuite tool addressed type test data generation keywords-test case generation java project data generation common trend unit testing essential future research direction bytecode level branch future research public class
