R. W. Butler and G. B. Finelli. The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Transactions on Software Engineering, 19(1):3--12, Jan. 1993.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....paths through the code, it is impossible, in typical practice to exhaustively test all the possible combinations. In non deterministic real time systems, the problem is compounded by the uncertainty in the execution times of various processes, the sequence of events, asynchronous interrupts, etc [Butler, 93] In general, the evaluation of intelligent systems (IS s) is broader than testing of non intelligent systems (NIS) A system that has intelligence should in general be able to perform under a wider range of operating conditions than one that does not have intelligence. In fact, it should learn ....

Butler, R.W., and Finelli, G.B. The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Trans. Softw. Eng 19, 1 (Jan. 1993), 3--12.

....with the inconvenience, expressed by Dijkstra, that testing can demonstrate the presence of bugs, but not their absence . For the most critical systems we must also accept the uncompromising Bayesian mathematics that no feasible amount of testing can provide assurance in the ultra critical region [12 14]. So it is an unfortunate fact that we cannot eliminate the possibility of runtime errors by testing alone. In practice, systems are typically tested to meet their functional requirements, up to a level of coverage required by a standard and any run time errors exposed by this process are dealt ....

Butler, Ricky W.; and Finelli, George B.: The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software. IEEE Transactions on Software Engineering, vol. 19, no. 1, Jan. 1993, pp 3-12.

....remains the dominant approach to verification, but testing is able to provide assurance only in the very simplest of systems. It has been shown that it is impossible to assess ultra high dependability using testing in a manner reminiscent of statistical sampling, a process known as life testing [19, 20]. The only viable alternative is to use formal verification, and case studies in the use of formal verification have been quite successful. However, presently formal verification has many limitations, such as floating point arithmetic and concurrent systems, that preclude its comprehensive and ....

Finelli, G.B., Butler, R.W.: The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software. IEEE Transactions on Software Engineering, pp. 3-12 (January 1993)

....assurance that all requirements have been met in a representative sample of operational contexts, and that full code coverage has been achieved, is considerable. For the most critical systems, is has been argued that testing alone can never be adequate to develop the level of confidence required[12]. An alternative approach is to develop the software in a language that can be analysed statically using tool support, allowing the software to be rigorously validated as the design and code is produced. This approach is particularly beneficial when applied during the design stage, when interface ....

Butler, Ricky W.; and Finelli, George B, The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software. IEEE Transactions on Software Engineering, 19(1):3-12, January 1993.

.... of errors, these models can make predictions about the future occurrence of failures (extrapolation) For ultra reliable systems (10 failures hour or less) it has been proven that these types of models cannot be used because a typical system would have to be tested for 115000 years or more [10][70] In addition, for typical testing environments it is very hard to reproduce the operational profiles for rare events. Taking these factors into account and considering the time required to restart the system for each test run, the failure rates that can be verified empirically are limited to ....

....typical testing environments it is very hard to reproduce the operational profiles for rare events. Taking these factors into account and considering the time required to restart the system for each test run, the failure rates that can be verified empirically are limited to about failures hour [10]. 2.4.3 Formal methods Just as traditional engineers can model their designs with different kinds of continuous mathematics, formal methods is an attempt to supply the software engineers with mathematical logic and discrete mathematics as modeling and verification tools. Formal methods can ....

[Article contains additional citation context not shown here]

Butler, R.W. and Finelli, G.B. The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Transactions on Software Engineering, (19): 3-12, January, 1993.

....by sampling the input space, i.e. life testing, or as a component whose structure permits modeling of its failure probability from its design. Unfortunately, the quantification of software dependability by life testing has been shown to be infeasible in general for safety critical systems [4, 12]. The reason is that an infeasible number of tests are required to establish a useful bound on the probability of failure in the ultra dependable range. The large number of tests derives from the number of combinations of input values that can occur. It is quite literally the case that for most ....

Butler, R. W.; Finelli, G. B. The infeasibility of quantifying the reliability of lifecritical real-time software. IEEE Transactions on Software Engineering, Jan. 1991, 19(1), pp. 3--12.

.... Approach: A Tutorial[8] High Level Design Proof of a Reliable Computing Platform [4] Formal Specification and Verification of a Fault Masking and Transient Recovery Model for Digital Flight Control Systems [7] The Infeasibility of Quantifying the Reliability of Life Critical Real Time Software [3] The following reference describes the automated reasoning system that supports the Framework: The PVS Specification Language [6] 00 0930319A Rev B 1.6, 5 May 1998 Secure Computing Corporation MTLS 3 A system is fault tolerant if it behaves like a fault free system even in the presence of ....

Ricky W. Butler and George B. Finelli. The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software. IEEE Transactions on Software Engineering, 19(1):3-- 12, January 1993.

....However, as trigger conditions occur randomly, software failures may appear to happen randomly as well. If the faulty software is never executed, the fault will never produce an error or failure. So, it is difficult to apply hardware oriented terms like failure rates to software reliability [Gordon91, Pfleeger92, Dunn86, Hecht86, Butler93]. What is generally meant by reliable software, is that each software process generates correct internal states and outputs when supplied with valid inputs or sequences thereof, and that the response to invalid inputs is pre defined. Software is correct if it operates as specified. 1 ....

....error each time these same conditions occur. Hence, it is the probability of encountering these specific conditions that determines the software reliability . Life testing for flight critical reliability levels of 10 9 [10E 9 issue comes from where ]per flight hour is impractical or infeasible [Butler93], large, complex. The test time would be so large that even parallel testing of 10 or 100 systems would not reduce this to acceptable levels [and may never excite the fault] This type of testing is also cost prohibitive, based on the price of such systems and of the required test environment. ....

Butler, R.W., Finelli, G.B.: "The Infeasibility of Quantifying the Reliability of Life-Critical RealTime Software", IEEE Trans. On Software Engineering, Vol. SE-19, No. 1, Jan. 1993, pp. 3-12

....to errors. Methods to improve system reliability by recovering from errors or by reducing errors cannot be assessed if there is not a way of measuring their benefits. 2 Avizienis calls these two approaches fault tolerance and fault avoidance, respectively. 1.2. OBJECTIVE 5 In a pivotal paper [BF93], Butler and Finelli argue that it is infeasible to quantify the impact of specification and design errors on system reliability in the ultra high reliability domain 3 . Butler s and Finelli s paper is a confirmation of an existing trend away from ad hoc and incremental methods to reduce or ....

R.W. Butler and G.B. Finelli, The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software, in: IEEE Transactions on Software Engineering, Vol. 19, No. 1, January 1993, pages 3-12.

....reliability levels which are commensurate with reasonable expectations. The probabilistic assessment 21 of software reliability to levels required in the safety critical applications, e.g. failure rate 10 Gamma9 h or 10 Gamma5 failure probability per demand, is currently out of reach [3], since random testing would require decades of testing before it could establish a reasonable statistical confidence in the reliability estimate. It is generally agreed that a practical current limit in the assessment of a failure rate before operational use lies in the range 10 Gamma2 h ....

R.W.Butler, G.B.Finelli, The infeasibility of quantifying the reliability of life-- critical real--time software, IEEE Trans. on Software Engineering, 19 (1) (1993) 3-12.

....functions) and information redundancy (error correcting codes) Generally, fault tolerance using redundancy depends on statistical independence among the failure occurrences. This is reasonable to assume for hardware component failures but less clear wrt. hardware or software design flaws [7]. Reliability goals for safety critical systems are often expressed quantitatively via the maximum allowed failure rate. For example, critical services of the Advanced Automation System (AAS, providing Air Traffic Control services) should be unavailable at most 3 seconds a year [9] To prevent ....

R. W. Butler and G. B. Finelli. The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Transactions on Software Engineering, 19(1):3--12, Jan. 1993.

....budget constraint. 4 Highly Reliable Software As already argued in section 3.2.1, the reliability r i of a (intensely pre tested) module version can be assumed to be close to one. Even for software on a moderate reliability level, only one failure in 10 3 Gamma 10 7 hours is expected (see [4]) Using the rough estimate that a program run takes one hour in the average, this results in an unreliability 1 Gamma r i of less than 0:001. So it makes sense to investigate how the models and optimization problem formulations of sections 2 3 behave in the limiting case where the ....

Butler, R. W., Finelli, G. B., "The infeasibility of quantifying the reliability of life-- critical software", IEEE Trans. Software Eng., vol. SE-19, 3 -- 2 (1993).

....domain: t i indicates the probability that, in a single test run, input x i is selected as a test case. For example, we may perform random testing under the operational distribution, i.e. we choose t = p. This seems to be the most commonly used technique of reliability testing (cf. e.g. [3] or [12] The relative frequency of unsuccessful runs under the operational distribution is an unbiased estimate of the unreliability 1 Gamma R( according to Nelson s definition. If t 6= p, this is not true anymore. We may, however, weight runs that produced failures by appropriate scores, ....

....low cost and high cost faults is especially drastic. We do not assert, however, that our approach can overcome the principal limitations to reliability estimation for ultra reliable software (failure rate 10 Gamma7 per hour) as they were exposed by Butler and Finelli in their lucid paper [3]. Finally, let us remark that reliability testing is an essential component of special modern development methods such as Cleanroom that put more emphasis on error prevention in early phases than on fault detection in late phases, and consider the testing process rather as a certification of ....

R. W. Butler and G. B. Finelli, "The infeasibility of quantifying the reliability of life--critical software", IEEE Trans. Software Eng., vol. SE-19, pp. 3--12, Jan. 1993.

.... a piece of software, it will manifest itself the first time the relevant condition occurs [Abbott 90] What allows the use of reliability as a measure of software quality is the fact that the software is embedded in a stochastic environment that generates input sequences to the software over time [Butler 91] Some of those inputs will result in software failures. Thus, reliability becomes a weighted measure of correctness, with the weights being dependent on the actual use of the software. Overall, different environments will result in different reliability values [Abbott 90] Much controversy ....

....in some modules can result in problems in other modules. When it comes to software reliability estimation, the only reasonable approach is to treat the software as a black box [Parnas 90] This approach is applicable to systems with low to moderate reliability requirements. However, according to [Butler 91] testing of software with a very high target reliability is prohibitively impractical. In addition, DO178B] states: methods for estimating the post verification probabilities of software errors were examined. The goal was to develop numerical requirements for such probabilities for ....

Ricky W. Butler and George B. Finelli, The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software, IEEE Transactions on Software Engineering, Vol. 19, No. 1, January 1991, pp. 3 -- 12.

....that faults are avoided completely during development. Similarly, fault detection techniques are imperfect. Research has shown, for example, that testing as an approach to verification cannot demonstrate sufficient levels of dependability because of the sheer number of tests that are required [3]. Building very small, simple software systems that achieve the extreme dependability necessary with safety critical applications has proven to be sufficiently challenging. The complexity of large systems involving characteristics such as real time operation and distributed processing is likely ....

Butler, R. W. and G. B. Finelli, "The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software," IEEE Transactions on Software Engineering, Vol. 19-1, pp. 3-12, January 1993.

....that DOLILU system requires demonstration of failure probability to be under 10 4 . Due to the criticality of the program, required confidence level should surpass 0.99. The state of the practice in software reliability engineering states that these reliability levels are practically achievable [But93, Law92]. In principle, software reliability can be quantified through program verification or statistical testing. The requirements specification for DOLILU system is written in English and no attempt has been made to formalize it in any form of mathematical notation. Furthermore, the size and the ....

R. W. Butler, G. B. Finelli, "The Infeasibility of Quantifying the Reliability of Life-Critical RealTime Software," IEEE Trans. Software Eng., Vol. 19, No. 1, Jan. 1993, pp. 3-12.

No context found.

Butler, Ricky W.; and Finelli, George B.: The Infeasibility of Quantifying the Reliability of LifeCritical Real-Time Software. IEEE Transactions on Software Engineering, vol. 19, no. 1, Jan. 1993, pp. 3--12.

No context found.

R. W. Butler and G. B. Finelli. The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Transactions on Software Engineering, 19(1):3--12, Jan. 1993.

No context found.

Ricky W. Butler and George B. Finelli. The infeasibility of quantifying the reliability of life-critical real-time software. Software Engineering, 19(1):3--12, 1993. Available at http: //citeseer.nj.nec.com/butler93infeasibility.html. 141

No context found.

R. W. Butler and G. B. Finelli. The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Transactions on Software Engineering, 19(1):3--12, Jan. 1993.

No context found.

R. W. Butler and G. B. Finelli. The infeasibility of quantifying the reliability of life-critical real-time software. IEEE TSE, 19(1):3-12, 1993.

No context found.

R. W. Butler and G. B. Finelli. The Infeasibility of Quantifying the Reliability of Life-Critical RealTime Software. IEEE Trans. on Software Engineering, 19(1):3--12, Jan. 1993.

No context found.

R. Butler and G. Finelli, "The infeasibility of quantifying the Reliability of Life-Critical Real-Time Software", IEEE Trans. on Soft. Engineering, Vol. 19, Nr. 1, Pp 3-12, Jan. 1993.

No context found.

R. W. Butler and G. B. Finelli. The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Transactions on Software Engineering, pages 3-12, 1993.

No context found.

Butler, R.W., G.B. Finelli, "The Infeasibility of Quantifying the Reliability of Life-Critical RealTime Software," IEEE Transactions on Software Engineering, Vol. 19, No. 1, Jan. 1993.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC