A briefer version of this paper with the same name has been published in

This paper summarizes several classes of software cost estimation models and techniques: parametric models, expertise-based techniques, learning-oriented techniques, dynamics-based models, regression-based models, and composite-Bayesian techniques for integrating expertise-based and regression-based models. Experience to date indicates that neural-net and dynamics-based techniques are less mature than the other classes of techniques, but that all classes of techniques are challenged by the rapid pace of change in software technology. The primary conclusion is that no single technique is best for all situations, and that a careful comparison of the results of several approaches is most likely to produce realistic estimates. 1.

Abstract—Effort estimation often requires generalizing from a small number of historical projects. Generalization from such limited experience is an inherently underconstrained problem. Hence, the learned effort models can exhibit large deviations that prevent standard statistical methods (e.g., t-tests) from distinguishing the performance of alternative effort-estimation methods. The COSEEKMO effort-modeling workbench applies a set of heuristic rejection rules to comparatively assess results from alternative models. Using these rules, and despite the presence of large deviations, COSEEKMO can rank alternative methods for generating effort models. Based on our experiments with COSEEKMO, we advise a new view on supposed “best practices ” in model-based effort estimation: 1) Each such practice should be viewed as a candidate technique which may or may not be useful in a particular domain, and 2) tools like COSEEKMO should be used to help analysts explore and select the best method for a particular domain. Index Terms—Model-based effort estimation, COCOMO, deviation, data mining. 1

We describe a method to quantify the value of investments in software systems. For that,

There are many goals for outsourcing information technology: for instance, cost reduction, speed to market, quality improvement, or new business opportunities. Based on

www.elsevier.com/locate/scico

Abstract: The purpose of this essay is to present a costing model for software maintenance and evolution based on a separation of fixed and variable costs. There has always been a problem in distinguishing between the maintenance activities covered by the standard maintenance fee and those charged extra to the user. Separating these two types of costs is essential to every maintenance operation to prevent costs from getting out of control. In this paper the author proposes a solution, which can lead to better cost estimations and a financially more stable maintenance operation. Particular emphasis is placed on a sharp division between work done to maintain the system functionality as it is and work done to enhance that functionality.

Many IT-metrics display large variation, time dependencies and noise, making it seemingly impossible to draw conclusions from them. Most of the software engineering literature proposed ways to stamp out this undesired behaviour, so that simple questions by management become simple to answer. In this paper we accepted that IT-metrics misbehave, in fact, we argued that large variation, time dependencies, and considerable noise are inherent to many IT-metrics. Many other fields know misbehaving metrics as well. These metrics range from the long-term temperature dynamics of beaver to the intra-tick graphs of the S&P500, their behaviour being sometimes even worse than our IT-metrics. We successfully applied the analysis methods common in other fields to software engineering questions. We illustrated our approach by solving a real-world problem. We answered the simple question by management whether a software process improvement program affecting 1500 IT-developers and business staff delivered its value. Moreover, we were able to predict the trends of important KPIs, like cost per function point, which enabled proactive steering and control. Our approach is not limited to this single question, but has a rich application potential to countless management and control issues concerning information technology.

We proposed a method to quantify the yield of an IT-investment portfolio in an environment of uncertainty and risk. For various common implementation scenarios such as growing demands during implementation without deadline extensions we showed how to monetize their impact on the net present value. Depending on the business case this can lead to higher or lower gains. We also took failure of projects within an IT-investment portfolio into account, by appraising the loss in case of failure, resulting in a more realistic yield. To provide maximal insight into this yield, we proposed to treat it as a stochastic variable. We explained how to infer various portfolio yield distributions: discrete, continuous, and cumulative distributions, leading to useful summaries such as box plots and histograms. We argued that these information-rich characterizations support decision makers in taking calculated risks, and provided insight in how to address IT-specific risks and what such risk mitigation may cost. We explained our approach by quantifying the expected yield of a small four project portfolio under uncertainty and risk, and we provided the results for a larger and realistic IT-investment portfolio. Keywords and Phrases: appraisal, IT-investment portfolio, risk-bearing IT-portfolio, uncertainty and risk, scenario analysis, IT-portfolio management, valuation, discounted

In this paper, we showed how to quantify the quality of IT forecasts based on Boehm’s cone of uncertainty and DeMarco’s Estimating Quality Factor. With these, we support decision making by providing critical information on IT forecasting quality to IT governors. We illustrated that plotting forecast to actual ratios against a predefined referential conical shape reveals potential biases, for instance political, involved with IT forecasting. The Estimating Quality Factor quantifies the deviation of forecasts from their actual value. Using simulations, we showed that the conical shape of Boehm’s cone is not caused by improved estimation, but can also be found when estimation accuracy decreases. We illustrated our approach by applying it to four real-world case studies (1741 projects, 12187 forecasts, 1059 million Euro). Finally, we surveyed benchmarks related to forecasting and proposed new benchmarks based on our extensive data. Most forecasting benchmarks in the literature turned out to have an unknown bias. As a consequence, we argued that such figures including Standish’s project success benchmarks are meaningless.