• Summary
• Related Documents
  • Active Bibliography
  • Co-citation
• Version History

Abstract:

The question \How much testing is enough? " has led many to explore code coverage metrics and structural testing. High branch coverage is common in unit testing, but can be a dicult goal to reach in system level testing. Before attempting to increase branch coverage, it is important to understand the tradeos between testing cost and additional fault detection. One part of this understanding is knowing which classes of faults are easily detected with structural testing methods, and which are not. To answer this question, we studied ninety eight post release software faults on a leading on-line transaction processing system. For each fault, we determined whether or not it had been covered in prerelease testing. We assigned each fault to a class, using a fault taxonomy from a similar study [SC92]. Finally, we examined which classes of faults were most prevalent, and which classes most often appear in covered branches. To quantify this, we dened a branch which was modied during a x as an \aected branch", and de ned \aected branch coverage " to be the percentage of aected branches that had been covered in testing. We dened the overall aected branch coverage to be the percentage of all aected branches that had been covered in testing. We measured the overall affected branch coverage for each fault class, and for all of the ninety-eight faults. Our results show that whether or not a fault appears in covered branches depends strongly on the fault class. Certain classes of faults had well over 50% overall aected branch coverage | increasing coverage would probably oer limited gains in fault detection for these classes. Overall, 60.3 % of all faults were in classes with high aected branch coverage, and the af-

Citations