DURAN,J.W.AND NTAFOS, S. C. 1984. An evaluation of random testing. IEEE Trans. Softw. Eng. SE-10, 4 (July), 438 -- 444.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....and an automatically generated list of all possible command response pairs. Stochastic specifications. It has been observed that random testing is not inferior to partition testing w.r.t. error detection if no increased probability of errors can be assigned to certain input data partitions [4]. This motivates test case specifications that lead to the randomized generation of all (symbolic) test sequences up to a certain length (up to some hundred commands) to reduce the number of sequences generated, we demanded that two test generated sequences di#ered at least to a given extent. ....

Duran, J. and S. Ntafos, An Evaluation of Random Testing, IEEE TSE SE-10 (1984), pp. 438--444.

....18 9 Average block coverage ( of test sets with fixed size . 19 10 Average effectiveness ( of test sets with fixed block coverage . 20 1 Introduction Random testing is a long standing testing technique and many researchers have studied its fault detection effectiveness [6, 7, 12, 17, 18]. The results of these studies are diverse. Some researchers [4, 6, 7, 13] conclude that random testing can be used to replace coverage based testing such as data flow and mutation testing. They make such conclusions based on the advantages of random testing such as reduced cost, high coverage in ....

....10 Average effectiveness ( of test sets with fixed block coverage . 20 1 Introduction Random testing is a long standing testing technique and many researchers have studied its fault detection effectiveness [6, 7, 12, 17, 18] The results of these studies are diverse. Some researchers [4, 6, 7, 13] conclude that random testing can be used to replace coverage based testing such as data flow and mutation testing. They make such conclusions based on the advantages of random testing such as reduced cost, high coverage in branch testing, and high coverage in mutation testing. Other researchers ....

[Article contains additional citation context not shown here]

J. W. Duran and S.C. Ntafos, "An evaluation of random testing," IEEE Trans. on Software Engineering, SE-10(7):438 444, July 1984.

....debuggers intuition, if indeed better in finding bugs than mere random choice of test cases, guides them to discover those bugs that contribute heavily to unreliability, so that the correction effort is cost effective. Several authors have published comparisons between classes of testing methods [1 9]. Experimental comparisons have been inconclusive so far. Among comparisons based on modelling, we think the most advanced so far is in [8, 9] to which we refer the reader for more extensive background material and whose analysis we seek to extend. Most previous studies of the effectiveness of ....

J. Duran and S. Ntafos, "An evaluation of random testing", IEEE TSE, 10, pp. 438-444, 1984.

....design approach differs from most other black box methods in that its basic test requirement is coverage of all valid n way test parameters combinations for tester defined values of n. A method related to our approach is random input testing and partition testing (see, e.g. Duran and Ntafos [9] and Hamlet and Taylor [13] The AETG approach differs from random testing by allowing the tester to define complex relationships between the test parameters. The tester can use the AETG constructs for relations, constraints and hierarchy to focus testing. The AETG test plans are far from ....

J. Duran and S. Ntafos, "An Evaluation of Random Testing," IEEE Tran. Software Eng., vol. 10, pp. 438-444, July 1984.

....But unfortunately, the assumption behind partition testing that all values within a subdomain are equivalent only represents the ideal, in practice this is extremely dicult to achieve. Recently, much research has been done to compare the e ectiveness of random testing and partition testing [3,4,5]. Their surprising result is that the two methods are of almost equal value, under assumptions that seem to favor partition testing. Duran and Ntafos [3] investigated the simulation model the probability of detecting at least one failure to compare the e ectiveness of random testing and ....

....is extremely dicult to achieve. Recently, much research has been done to compare the e ectiveness of random testing and partition testing [3,4,5] Their surprising result is that the two methods are of almost equal value, under assumptions that seem to favor partition testing. Duran and Ntafos [3] investigated the simulation model the probability of detecting at least one failure to compare the e ectiveness of random testing and partition testing. In their studies, they drew the mathematical formulas of the probability of detecting failures by random testing and partition testing, and ....

[Article contains additional citation context not shown here]

J.W. Duran and S.C. Ntafos, \ An Evaluation of Random Testing," IEEE Trans. Software Engineering, vol.10, pp.438-444, July 1984.

....consequently any ATG. For this reason, at least, further research in the test data adequacy area is needed. So, as testing strategies cannot be evaluated against formal criteria it is tempting to try to compare them through the use of mathematical models. Unfortunately, it has been found difficult [2, 33, 34] to construct mathematical models for the different testing strategies available even for random testing that could be used to assess their performance. The human factors involved in the software development process are partially to blame as they are, indeed, difficult to formalise. It is ....

....is intuitively the poorest strategy of selection of inputs to test a program. Early books on software testing [18]do not discuss it at all other than to dismiss it. A study by Hamlet [2] on the efficiency of random testing tends to dismiss this intuitive view. It follows a report from Ntafos [33]. Partition testing strategies see below have been examined in comparison to random testing. In [34] Weyuker reviews the results of Hamlet and clarifies them. Where an experimental approach was used by Hamlet and Ntafos, Weyuker uses an analytic approach. Her results are more in tune with the ....

J. Duran and S. Ntafos, "An evaluation of random testing," IEEE Transactions on Software Engineering, vol. 10, pp. 438--444, July 1984. 200

....testing cf. Myers [18] verdict that probably the poorest [testing] methodology is random input testing , the judgment on the random testing approach became more positive during the eighties. In theoretical research, a surprising defense of random testing was given by Duran and Ntafos [5], who put the problem on a well defined formal base by comparing random testing (i.e. selection of test inputs randomly from the whole input domain) to partition testing (i.e. dividing the input domain into non overlapping subdomains and selecting one test input from each subdomain) Under ....

....the distribution of the failure rates could be measured empirically to any desired degree of accuracy in a quite objective way. 6 class only consists of the two indicated programs. Then the expected failure rates for the subdomains are 1 = 0:05 and 2 = 0. 2 In Duran s and Ntafos paper [5], there is already an implicit probabilistic consideration of the failure rates in the subdomains, since in their simulation experiments, the authors have assigned failure rates to the subdomains according to carefully selected distributions on the interval [0; 1] Nevertheless, their results are ....

[Article contains additional citation context not shown here]

J. W. Duran, S. C. Ntafos, "An evaluation of random testing", IEEE Trans. Software Eng., vol. SE-10, pp. 438--444, 1984.

.... criteria that relate either to a model of the program (structural testing) or a model of its functionality (functional testing) Beizer 1990] Given a model and a criterion, there are two principles for generating test inputs : deterministic (see e.g. Beizer 1990] and probabilistic (see e.g. [Duran Ntafos 1984, Thvenod Fosse et al. 1995] In the deterministic approach, test inputs are selected from the input domain (generally by hand) in accordance with the criterion. In statistical testing ( Thvenod Fosse et al. 1995] test patterns are selected according to a defined probability distribution on the ....

J. W. Duran and S. C. Ntafos, "An evaluation of random testing", IEEE Transactions on Software Engineering, SE-10 (4), pp.438-44, 1984.

....consequently any ATG. For this reason, at least, further research in the test data adequacy area is needed. So, as testing strategies cannot be evaluated against formal criteria it is tempting to try to compare them through the use of mathematical models. Unfortunately, it has been found difficult [28, 29, 30] to construct mathematical models for the different testing strategies available even for random testing that could be used to assess their performance. The human factors involved in the software development process are partially to blame as they are, indeed, difficult to formalise. It is ....

....the poorest strategy of selection of inputs to test a program. Early books on software testing [16] do not discuss it at all otherwise than for dismissing it. A recent study by Hamlet [29] on the efficiency of random testing tends to dismiss this intuitive view. It follows a report from Ntafos [28]. Partition testing strategies see below have been examined in comparison to random testing. In [30] Weyuker reviews the results of Hamlet and clarifies them. Where an experimental approach was used by Hamlet and Ntafos, Weyuker uses an analytic approach. Her results are more in tune with the ....

J. Duran and S. Ntafos, "An evaluation of random testing," IEEE Transactions on Software Engineering, vol. 10, pp. 438--444, July 1984.

....to testing . While the type of testing that we use may be ad hoc, we do seem to be able to find bugs in real programs. Our view is that random testing is one tool (and an easy one to use) in a larger software testing toolkit. An early paper on random testing was published by Duran and Ntafos[3]. In that study, test inputs are chosen at random from a predefined set of test cases. The authors found that random testing fared well when compared to the standard partition testing practice. They were able to track down subtle bugs easily that would otherwise be hard to discover using ....

J. W. Duran and S.C. Ntafos, "An Evaluation of Random Testing", IEEE Transactions on Software Engineering SE-10, 4, July 1984, pp. 438-444.

....for path coverage and the domain testing strategy. In the case of disjoint subdomains, the testing strategy is specifically referred to as partition testing. Intuitively speaking, partition testing should be more effective in revealing program errors than random testing. However, Duran and Ntafos [2] as well as Hamlet and Taylor [3] observed in their simulation investigations that there was only a marginal difference in effectiveness between the two methods. Random testing might even be more cost effective than partition testing when the partitioning and associated costs of partition testing ....

Duran, J W and Ntafos, S C, `An evaluation of random testing' IEEE Transactions on Software Engineering Vol 10 No 4 (1984) pp 438--444.

....of the effectiveness of testing techniques have used the failurefinding probability, the probability that a testset will detect at least one failure, as a measure of effectiveness. This measure was used in simulations comparing partition testing techniques to random testing by Duran and Ntafos [8], and Hamlet and Taylor [12] in analytical treatments by Weyuker and Jeng [15] and Chen and Yu [4] in analytical comparisons of various testing techniques by Frankl and Weyuker [10] and in experimental comparisons by Frankl and Weiss [9] and Mathur and Wong [23] The expected number of ....

....great variation, both in the care with which the method is defined and applied and in the results [11] Neither side has any real claim to establishing its case. Analysis of partition testing. A number of careful theoretical studies have compared random testing with debug ( partition ) testing [8, 12, 15, 22, 4, 5]. The original motivation for these studies was a belief that random testing might be a real alternative to partition testing for finding failures. However, no such conclusive result was obtained. Although random testing is a surprisingly good competitor for partition testing, it is seldom better, ....

[Article contains additional citation context not shown here]

J. Duran and S. Ntafos. An evaluation of random testing. IEEE Trans. on Soft. Eng., 10:438--444, 1984.

....programming languages. It may seem that our approach, using randomly generated test data, is naive in the extreme in comparison. But this is not so: in a classic paper, Duran and Ntafos showed by statistical reasoning that random testing is almost as effective as intelligent choice of test data [4] and it is much easier. Random testing is still a respectable test method, and indeed is implemented in commercial products, even if the focus of current work is elsewhere. As far as we are aware, we are the first to apply automatic testing to functional programs. The close link we establish ....

J. Duran and S. Ntafos. An evaluation of random testing. In IEEE Transactions on Software Engineering, volume SE-10, July 1984.

....to random testing. This is because random test selection may miss certain special test cases which are effective in detecting faults. Information obtained from these different testing strategies is not used by reliability growth models. Another point worth noting is that even though a few studies [9, 11] reported that random testing can be as efficient as other testing strategies, this is still debatable. To verify whether a non random test set is indeed better, we need to measure its fault detection efficiency. Voas et al. 28] and Bertolino and Strigini [1] defined testability as the ....

J. W. Duran and S. C. Ntafos, "An evaluation of random testing," IEEE Trans. on Software Engineering, SE-10(7):438-444, July 1984.

....estimate the expected complexity of straight line code. 1 Introduction Program testing is an important subfield of the field of software engineering. Much work has been done in finding methods for selecting test data [GG75, MH81, DMMP87] and in evaluating different testing methodologies [Bud81, DN84, BS87, Ham89] A related area of software engineering is the study of software complexity. Considerable research has been done in this area as well to devise software complexity measures [Hal77, McC76, WHH79] and to compare various measures for their effectiveness [Wey88, Tia92, TZ92] Our work ....

Duran, J.W. and Ntafos, S.C. (1984), An Evaluation of Random Testing, IEEE Trans. Software Engrg., SE-10(4):438--444.

....likely to hold when the all uses criterion is used during test set minimization. 123 8. EFFECT OF SIZE AND BLOCK COVERAGE ON THE FAULT DETECTION EFFECTIVENESS Random testing has existed for a long time as a testing technique. Several researchers have studied its fault detection effectiveness [21, 33, 60, 78, 85]. The results of these studies are diverse. Some researchers [14, 21, 33, 62] tend to conclude that random testing can be used to replace coverage based testing such as data flow and mutation testing. They make such conclusions based on the advantages of random testing such as reduced cost, high ....

....123 8. EFFECT OF SIZE AND BLOCK COVERAGE ON THE FAULT DETECTION EFFECTIVENESS Random testing has existed for a long time as a testing technique. Several researchers have studied its fault detection effectiveness [21, 33, 60, 78, 85] The results of these studies are diverse. Some researchers [14, 21, 33, 62] tend to conclude that random testing can be used to replace coverage based testing such as data flow and mutation testing. They make such conclusions based on the advantages of random testing such as reduced cost, high coverage on branch testing [78] and high coverage on mutation testing [64] ....

[Article contains additional citation context not shown here]

J. W. Duran and S. C. Ntafos, "An evaluation of random testing," IEEE Trans. on Software Engineering SE-10 (1984), no. 7, 438--444.

....basis functions. This gives some of our results a flavor similar to those that concern subdomain testing. Subdomain testing embodies the idea that software testing might be more effective if the set of possible program inputs were partitioned, or subdivided in some other way (cf. Goodeno. 75] Duran 84] Hamlet 88] Weyuker 91] Generally, the aim of this procedure is not to achieve profile independence, but rather to ensure that the program is tested thoroughly. However, it has also been suggested that partitions of the input set might be used to make profile independent measurements of ....

Duran, J., Ntafos, S., "An Evaluation of random testing," IEEE Transactions on Software Engineering Vol. SE-10, pp. 438-444, July, 1984.

....a role in the development process. 2.3. Example: simple pac analysis. One of the simplest forms of pac analysis involves a single program whose quality is to be evaluated by random testing. The technique presented in this example has appeared often in software testing literature, notably in [Duran 84] 2 and [Hamlet 87] 1 We assume that the program behaves deterministically, and that there are certain inputs for which it fails. The probability that the user will execute the program with one of these failure causing inputs is the program s probability of failure, and we use it as ....

....the software engineering literature that relate to pac analysis. In some papers, like [Hamlet 87] and [Voas 95] the goal is to get a high confidence that the program being tested has high quality. This is what was illustrated by the example in section 2.3. Another approach, used for example in [Duran 84] and [Weyuker 91] is to try to assess the probability that testing will miss faults in a program. These approaches are closely related to pac analysis because the probability of not missing a fault is just the confidence factor that a pac framework seeks to provide. Although the papers just ....

[Article contains additional citation context not shown here]

Duran, J., Ntafos, S., "An Evaluation of random testing," IEEE Transactions on Software Engineering Vol. SE-10, pp. 438-444, July, 1984.

....dictates. It is therefore unexpected that random testing can compete with systematic testing in its ability to expose failures. But that is exactly what studies have shown: under assumptions not unfavorable to systematic methods, they are not much better at finding failures than is random testing (Duran, 1984), Hamlet, 1990) The comparison of systematic methods with random testing is a useful exercise in its own right, since it has the potential for evaluating the systematic method to the random testing standard. Systematic methods are often intuitively appealing, but their significance is in doubt, ....

Duran, J. and S. Ntafos, An evaluation of random testing, IEEE Trans. Software Eng. SE-10 (July, 1984), 438-444.

....Let D be the input domain and consider a partition of D into disjoint subdomains D 1 , D k . The model associates a probability p i and a failure rate J i with each subdomain. The partition testing model has been used to compare partition strategies with random testing. Early studies [1,2] used the probability of detecting at least one failure as a measure of effectiveness and found little difference between the performance of random and partition testing (indicating that random testing may be more costeffective) The model was extended in [6] by introducing costs (c i is the cost ....

....they produce no benefit. We introduced the three theta (classify failure rates into three groups and use the group means as estimates) and the three two (combine three theta with two cost) strategies to allow for better modeling of near homogeneous failure rate distributions like those used in [1,2]. This reduces, but does not eliminate, the wasted allocation of test cases to high cost subdomains that have zero failure rates. Further insight is obtained by looking at best and worst case scenarios for the various strategies. The best case scenarios have the estimates used in place of the c i ....

Duran, J. and S. Ntafos, "An Evaluation of Random Testing," IEEE Trans. on Software Engineering, Vol. SE-10, No. 4, July

No context found.

DURAN,J.W.AND NTAFOS, S. C. 1984. An evaluation of random testing. IEEE Trans. Softw. Eng. SE-10, 4 (July), 438 -- 444.

No context found.

J. W. Duran and S. C. Ntafos. `An evaluation of random testing', IEEE Trans. Software Engineering, 10, (4) 438--444 (1984).

No context found.

J. Duran and S. Ntafos, "An evaluation of random testing", IEEE Transactions on Software Engineering, 10 (4), pp. 438-44, 1984.

No context found.

Duran&Ntafos84 J. W. Duran, S. C. Ntafos, An evaluation of random testing, IEEE Trans. Soft. Eng., SE-10, July 1984, pp. 438 - 444.

No context found.

J.W. Duran and S.C. Ntafos, "An Evaluation of Random Testing," IEEE Trans. Software Eng., vol. SE-10, pp. 438-444, 1984.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC