L. Briand, K. El Emam, and S. Morasca, "Theoretical and Empirical Validation of Software Product Measures," Technical Report ISERN-95-03, Fraunhofer Inst. for Experimental Software Eng., Germany, 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... demonstrating that the product metric is associated with some important external metric (such as measures of maintainability or reliability) These are also commonly referred to as theoretical and empirical validation respectively [73] and procedures for achieving both are described in [15]. Our focus in this paper is empirical validation. 2 Product metrics are of little value by themselves unless there is empirical evidence that they are associated with important external attributes [65] The demonstration of such a relationship can serve two important purposes: early ....

L. Briand, K. El Emam, and S. Morasca: "Theoretical and Empirical Validation of Software Product Measures". International Software Engineering Research Network, Technical Report ISERN-95-03, 1995.

....the benefit of any theoretical validation. Fenton [9] writes that measures must be validated in the narrow sense using measurement theory. Fenton s narrow validation is required to establish the scale of the measure in order to know which statistics can be legitimately applied. Briand, et al. [3] write that measurement theory, while valid for the structured paradigm, does not migrate to the object oriented paradigm. Zuse [30] writes that the Dempster Shafer Function of Belief allows us to substitute set theory for the intensive structure of measurement theory to validate measures in the ....

....of the concatenated entities must be preserved by the mapping to the formal relationship. Note that addition is assumed to be the concatenation operator. Criticism of the Extensive Structure Recent work has questioned the applicability of the extensive structure to object oriented measures [3], 4] 5] 29] and [30] Particularly important is the question: must the measurement of an entity formed by the concatenation of two modules equal the sum of the measurements of the independent modules before concatenation, i.e. let b1, b2, be the formal measures associated with a1, a2; is it ....

Briand, Lionel C., Khaled El Eman, and Sandro Morasca, "Theoretical and Empirical Validation of Software Product Measures", International Software Engineering Research Network technical report #ISERN-95-03, 1995a.

....order for the metric to be useful, there must be sufficient variation in the measurement of different entities to distinguish among them. Recent work has questioned the applicability of the extensive structure (used to validate metrics on the interval and ratio scales) to object oriented measures [3] [4] 5] 25] In particular, it has been argued that the set theory union structure test can be used. The set theory union structure is a weaker test than the extensive structure test, i.e. the extensive structure dominates the set theory union structure. Then, if the extensive structure fails ....

....for a scale above the level of the ordinal scale [25] We will call this the ratio scale. The question really is: if a measure is ordinal but fails the extensive structure, is the measure strictly ordinal or would much valuable data be lost by not considering the measure as a higher order scale [3] Parametric statistics have been shown to be more robust under the violation of the assumption of scale than nonparametric statistics [4] Therefore, there are two reasons to extend the scale of a measure to a higher level. The metric may be more powerful than the ordinal scale will reflect and ....

L. C. Briand, K. El Eman, and S. Morasca, "Theoretical and empirical validation of software product measures," International Software Engineering Research Network Technical Report #ISERN-95-03, 1995.

.... of object oriented metrics in numerous application domains should only take place if the metrics can be shown to be theoretically valid, in the sense that they accurately measure the attributes of software which they were designed to measure [1, 2, 3, 4] and have also been validated empirically [3, 4, 5]. Without such a basis, we will not be able to draw meaningful conclusions from analyses of metrics data. In this paper, we consider a set of metrics for object oriented design called the MOOD metrics [6, 7, 8, 9] from a measurement theory viewpoint, and then consider their empirical evaluation ....

L. Briand, K. El Emam, and S. Morasca, "Theoretical and empirical validation of software metrics," ISERN Technical Report 95-03, 1995.

....can be reused across environments and projects. Whereas the research underlying the validation of software product measures as internal attributes of software quality is not novel, it is only within the past few years that researchers have begun to worry about a rigorous and local validation [5, 9, 17, 25]. Predictive models are very attractive to build but they can be a waste of time if we rely on false assumptions instead of building a local process for selecting valid predictors. Acknowledgments We would like to thank the students from the University of Bari for providing the fault data used in ....

Briand, L., El Eman, K., and Morasca, S., Theoretical and empirical validation of software product measures, ISERN-95-03, International Software Engineering Research Network, 1995.

.... take into account the presence of interactions among independent variables through, for example, the specification of multiplicative terms in a regression equation [32] Typically, additional data analysis problems have to be addressed such as outlier analysis [1] or the statistical power [14] of the study. Briand and Wuest [12] provide detailed guidelines on these matters and an empirical validation procedure for software product measures. Figs. 2 and 3 show that most of the outputs (e.g. abstractions, measures) of the steps defined above are reusable. They should be packaged and ....

....as coefficients that are a part of regression models, for which standard tests are usually used. Since there is extensive literature on the empirical validation of software measures subject, we will not discuss this step any further. For instance, the interested reader may consult [12] 38] [14] for a detailed discussions of issues related to the empirical validation of software measures. 8IDENTIFYING PROBLEMS One of the strengths of a well defined measure definition approach is the support it can provide in locating and identifying the causes of problems. Based on our application ....

[Article contains additional citation context not shown here]

L. Briand, K. El Emam, and S. Morasca, "Theoretical and Empirical Validation of Software Product Measures," Technical Report ISERN-95-03, Fraunhofer Inst. for Experimental Software Eng., Germany, 1995.

.... demonstrating that the product metric is associated with some important external metric (such as measures of maintainability or reliability) These are also commonly referred to as theoretical and empirical validation respectively [72] and procedures for achieving both are described in [15]. Our focus in this paper is empirical validation. Product metrics are of little value by themselves unless there is empirical evidence that they are associated with important external attributes [64] The demonstration of such a relationship can serve two important purposes: early ....

L. Briand, K. El Emam, and S. Morasca: "Theoretical and Empirical Validation of Software Product Measures". International Software Engineering Research Network, Technical Report ISERN-95-03, 1995.

.... analysis, regression techniques [L80] are often used in the context of cross sectional designs whereas other types of analyses are typically used for controlled experiments (e.g. ANOVA [IN87] However, similar problems have to be addressed such as outlier analysis [BP95] or statistical power [BEM95] of the study. Figures 1 2 show that most of the outputs (e.g. abstractions, measures) of the steps defined above are reusable. They should be packaged and stored so that they can be efficiently and effectively reused, thus reducing the cost of measurement in an organization [BR88] In a ....

....algorithms, and their ease of interpretation. Different types of data and application purposes may warrant the use of different modeling techniques. Since there is extensive literature on the subject, we will not discuss this step any further. For instance, the interested reader may consult [S92, BEM95] for a detailed discussions of issues related to the empirical validation of software measures. 8. Identifying Problems One of the strengths of a well defined measure definition approach is the support it can provide in locating and identifying the causes of problems that may arise. Based on our ....

[Article contains additional citation context not shown here]

L. Briand, K. El Emam, S. Morasca, "Theoretical and Empirical Validation of Software Product Measures", Technical Report ISERN-95-03, Fraunhofer Institute for Experimental Software Engineering, Germany, 1995.

....However, such a decision should be made carefully because such a result may also be due to: the use of an inadequate modeling technique, e.g. a linear regression with an underlying exponential relationship. the sampling of a too small dataset leading to the lack of statistical power [BEM95b] If the expected impact is observed, the identified relationships may be used to build new or more reliable models for project management, quality assurance, etc. For control and change purposes, assuming the variation factors have already shown to be of some impact, the analysis concentrates ....

L. Briand, K. El Emam, S. Morasca. "Theoretical and Empirical Validation of Software Product Measures." ISERN technical report 95-03, 1995. ISERN-96-05

....hypothesis which drives measurement. The hypothesis will be of the form Internal attribute cohesion (as measured by the cohesion measures to be defined) has a causal effect on external quality attribute Y. The external attribute Y could be maintainability, reliability etc. As discussed in [BEM95], we believe that product measures by themselves, no matter how well defined, are not guaranteed to capture any relevant phenomenon regarding the quality of the system under study. It must be shown empirically that they are related to some external quality attribute of interest. In other words, it ....

L. Briand, K. El Emam, S. Morasca, "Theoretical and Empirical Validation of Software Product Measures ", Technical Report, Centre de Recherche Informatique de Montréal, 1995

No context found.

L. Briand, K. El Emam, S. Morasca, "Theoretical and Empirical Validation of Software Product Measures ", Technical Report ISERN-95-03, 1995.

....underlying hypothesis which drives measurement. The hypothesis will be of the form Internal attribute coupling (as measured by the coupling measures defined) has a causal effect on external quality attribute Y. The external attribute Y could be maintainability, reliability, etc. As discussed in [BEM95], we believe that product measures by themselves, no matter how well defined, are not guaranteed to capture any relevant phenomenon regarding the quality of the system under study. It must be shown empirically that they are related to some external system quality attribute of interest. In other ....

....be empirically validated with respect to the external quality attribute of interest specified in the measurement goal. We believe that measurement of internal product attributes are not meaningful in isolation, but only if they capture relevant external quality attributes; for full details, see [BEM95]. Existing measures have been classified according to the options available for each criterion of the framework. This classification allows existing measures to be compared and their potential use identified. The classification has shown that some particular options of the framework criteria ....

L. Briand, K. El Emam, S. Morasca, "Theoretical and Empirical Validation of Software Product Measures ", Technical Report, Centre de Recherche Informatique de Montréal, 1995.

....interactions due to run time virtual function bindings are not captured. Validation Strategy Logistic regression analysis [14] is used here as a means to empirically validate the coupling measures we defined. When validating a product measure, there are at least four questions to be considered [6]: 1) is the measure adequately capturing the attribute it purports to measure (i.e. construct validity) 2) is the attribute itself well defined based on an explicit empirical model (i.e. empirical relational system) 3) is there any empirical evidence supporting the underlying hypotheses of ....

L. Briand, K. El Emam, S. Morasca. "Theoretical and Empirical Validation of Software Product Measures." ISERN technical report 95-03, 1995.

....products must be used to validate the hypotheses upon which the measures are built. The procedure to follow for experimental validation varies significantly depending on the purpose of measurement (e.g. evaluation vs. prediction) the type of data collected, the amount of data collected, etc [1]. Consistent with the stated measurement goals, we want to better understand the relationship if any between the defined measures and the changes and effort associated with TRIO specification classes. Thus, we would like to validate the measures underlying hypotheses and thus better ....

....predictive models based exclusively on them. The statistical significance of a predictor only shows that the predictor has an impact on the dependent variable (specification effort or changes, in our case) and that it is a relevant quality or productivity factor to be monitored and or controlled [1]. To build an accurate predictive model, one has to find a set of predictors, which, when used together, explain a large percentage of the variation of the dependent variable in the data set. As we will see, we also found other fairly good predictors that can be used with the number of axioms of a ....

L. Briand, K. El Emam, S. Morasca, "Theoretical and Empirical Validation of Software Product measures", ISERN TR 95-03, October 1995.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC