Results 1 - 10
of
10
An Orchestrated Survey on Automated Software Test Case Generation
, 2013
"... Test case generation is among the most labour-intensive tasks in software testing and also one that has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has also been one of the most active topics in the research on software testing for several decades, ..."
Abstract
-
Cited by 21 (2 self)
- Add to MetaCart
Test case generation is among the most labour-intensive tasks in software testing and also one that has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has also been one of the most active topics in the research on software testing for several decades, resulting in many different approaches and tools. This paper presents an orchestrated survey of the most prominent techniques for automatic generation of software test cases, reviewed in self-standing sections. The techniques presented include: (a) structural testing using symbolic execution, (b) model-based testing, (c) combinatorial testing, (d) random testing and its variant of adaptive random testing, and (e) search-based testing. Each section is contributed by world-renowned active researchers on the technique, and briefly covers the basic ideas underlying the technique, the current state of art, a discussion of the open research problems, and a perspective of the future development in the approach. As a whole, the paper aims at giving an introductory, up-to-date and (relatively) short overview of research in automatic test case generation, while ensuring comprehensiveness and authoritativeness.
Optimizing the Automatic Test Generation by SAT and SMT Solving for Boolean Expressions
"... Abstract—Recent advances in propositional satisfiability (SAT) and Satisfiability Modulo Theories (SMT) solvers are increasingly rendering SAT and SMT-based automatic test generation an attractive alternative to traditional algorithmic test generation methods. The use of SAT/SMT solvers is particula ..."
Abstract
-
Cited by 4 (2 self)
- Add to MetaCart
(Show Context)
Abstract—Recent advances in propositional satisfiability (SAT) and Satisfiability Modulo Theories (SMT) solvers are increasingly rendering SAT and SMT-based automatic test generation an attractive alternative to traditional algorithmic test generation methods. The use of SAT/SMT solvers is particularly appealing when testing Boolean expressions: These tools are able to deal with constraints over the models, generate compact test suites, and they support fault-based test generation methods. However, these solvers normally require more time and greater amount of memory than classical test generation algorithms, limiting their applicability. In this paper we propose several ways to optimize the process of test generation and we compare several SAT/SMT solvers and propositional transformation rules. These optimizations promise to make SAT/SMT-based techniques as efficient as standard methods for testing purposes, especially when dealing with Boolean expressions, as proved by our experiments. I.
Dealing with Constraints in Boolean Expression Testing
"... Abstract—When testing a Boolean expression, one should consider also the constraints among the variables contained in it. Constraints model interdependence among the conditions in the expressions. Only tests that satisfy the constraints. i.e. valid tests, are really useful and can be applied to test ..."
Abstract
-
Cited by 4 (2 self)
- Add to MetaCart
(Show Context)
Abstract—When testing a Boolean expression, one should consider also the constraints among the variables contained in it. Constraints model interdependence among the conditions in the expressions. Only tests that satisfy the constraints. i.e. valid tests, are really useful and can be applied to test the expression. We present three ways to deal with such constraints: (1) ignoring them during test generation and removing invalid tests later, (2) including them in the expression as conjoint and again removing invalid tests later, and (3) considering them during the test generation process in order to generate only valid tests from the start. We introduce a general framework in which the three policies are implemented and compared over a set of Boolean expressions commonly used as benchmarks. Although the third policy requires a constraints solving technique for actual test generation, it presents several benefits: it generates smaller test suites and it may require less time for tests generation. Moreover, ignoring the constraints during test generation can reduce the fault detection capability of the tests. [Note: this version corrects several errors in the original paper- the author] I.
CITLAB: a Laboratory for Combinatorial Interaction Testing
"... Abstract—Although the research community around combinatorial interaction testing has been very active for several years, it has failed to find common solutions on some issues. First of all, there is not a common abstract nor concrete language to express combinatorial problems. Combinatorial testing ..."
Abstract
-
Cited by 4 (4 self)
- Add to MetaCart
(Show Context)
Abstract—Although the research community around combinatorial interaction testing has been very active for several years, it has failed to find common solutions on some issues. First of all, there is not a common abstract nor concrete language to express combinatorial problems. Combinatorial testing generator tools are strongly decoupled making difficult their interoperability and the exchange of models and data. In this paper, we propose an abstract and concrete specific language for combinatorial problems. It features and formally defines the concepts of parameters and types, constraints, seeds, and test goals. The language is defined by means of XTEXT, a framework for the definition of domain-specific languages. XTEXT is used to derive a powerful editor integrated with eclipse and with all the expected features of a modern editor. Eclipse is also used to build an extensible framework in which test generators, importers, and exporters can be easily added as plugins. Index Terms—Combinatorial testing model, domain-specific language, eclipse, XTEXT. I.
Answer-Set Programming as a new Approach to Event-Sequence Testing
"... Abstract—In many applications, faults are triggered by events that occur in a particular order. Based on the assumption that most bugs are caused by the interaction of a low number of events, Kuhn et al. recently introduced sequence covering arrays (SCAs) as suitable designs for event sequence testi ..."
Abstract
-
Cited by 3 (2 self)
- Add to MetaCart
(Show Context)
Abstract—In many applications, faults are triggered by events that occur in a particular order. Based on the assumption that most bugs are caused by the interaction of a low number of events, Kuhn et al. recently introduced sequence covering arrays (SCAs) as suitable designs for event sequence testing. In practice, directly applying SCAs for testing is often impaired by additional constraints, and SCAs have to be adapted to fit application-specific needs. Modifying precomputed SCAs to account for problem variations can be problematic, if not impossible, and developing dedicated algorithms is costly. In this paper, we propose answer-set programming (ASP), a wellknown knowledge-representation formalism from the area of artificial intelligence based on logic programming, as a declarative paradigm for computing SCAs. Our approach allows to concisely state complex coverage criteria in an elaboration tolerant way, i.e., small variations of a problem specification require only small modifications of the ASP representation. Keywords-event-sequence testing; combinatorial interaction testing; answer-set programming. I.
Efficient combinatorial test generation based on multivalued decision diagrams
- in Hardware and Software: Verification and Testing, Haifa Verification Conference HVC 2014, ser. Lecture Notes in Computer Science
, 2014
"... Abstract. Combinatorial interaction testing (CIT) is an emerging test-ing technique that has proved to be effective in finding faults due to the interaction among inputs. Efficient test generation for CIT is still an open problem especially when applied to real models having meaningful size and cont ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
Abstract. Combinatorial interaction testing (CIT) is an emerging test-ing technique that has proved to be effective in finding faults due to the interaction among inputs. Efficient test generation for CIT is still an open problem especially when applied to real models having meaningful size and containing many constraints among inputs. In this paper we present a novel technique for the automatic generation of compact test suites starting from models containing constraints given in general form. It is based on the use of Multivalued Decision Diagrams (MDDs) which prove to be suitable to efficiently support CIT. We devise and experiment several optimizations including a novel variation of the classical greedy policy normally used in similar algorithms. The results of a thorough comparison with other similar techniques are presented and show that our approach can provide several advantages in terms of applicability, test suite size, generation time, and cost. 1
Learning Combinatorial . . . using Hyperheuristic Search
, 2013
"... Two decades of bespoke Combinatorial Interaction Testing (CIT) algorithm development have left software engineers with a bewildering choice of configurable system testing techniques. This paper introduces a single hyperheuristic algorithm that earns CIT strategies, providing a single generalist appr ..."
Abstract
- Add to MetaCart
Two decades of bespoke Combinatorial Interaction Testing (CIT) algorithm development have left software engineers with a bewildering choice of configurable system testing techniques. This paper introduces a single hyperheuristic algorithm that earns CIT strategies, providing a single generalist approach. We report experiments that show that our algorithm competes with known best solutions across constrained and unconstrained problems. For all 26 real world subjects and 29 of the 30 constrained benchmark problems studied, it equals or improves upon the best known result. We also present evidence that our algorithm's strong generic performance is caused by its effective unsupervised learning. Hyperheuristic search
Design and Development of an Extensible Test Generation Tool based on the Eclipse Rich Client Platform
"... Abstract. In aspiration of automated software testing, a common task is the derivation of test cases from models. The wealth of di erent test criteria, model formalisms, and testing strategies makes reusability of such test generation tools a very challenging task. Leveraging the exibility o ered by ..."
Abstract
- Add to MetaCart
(Show Context)
Abstract. In aspiration of automated software testing, a common task is the derivation of test cases from models. The wealth of di erent test criteria, model formalisms, and testing strategies makes reusability of such test generation tools a very challenging task. Leveraging the exibility o ered by the Eclipse Rich Client Platform, we present a new test generation tool that achieves reusability by abstracting from speci c details of the test generation, and by matching these features with Eclipse extensions. The resulting tool allows the con guration of di erent backends for extracting tests from models, input languages, test strategies, and test criteria via plug-ins. 1
How to Optimize the Use of SAT and SMT Solvers for Test Generation of Boolean Expressions
"... In the context of automatic test generation, the use of propositional satisfiability (SAT) and Satisfiability Modulo Theories (SMT) solvers is becoming an attractive alternative to traditional algorithmic test generation methods, especially when testing Boolean expressions. The main advantages are t ..."
Abstract
- Add to MetaCart
(Show Context)
In the context of automatic test generation, the use of propositional satisfiability (SAT) and Satisfiability Modulo Theories (SMT) solvers is becoming an attractive alternative to traditional algorithmic test generation methods, especially when testing Boolean expressions. The main advantages are the capability to deal with constraints over the inputs, the generation of compact test suites, and the support for fault detecting test generation methods. However, these solvers normally require more time and a greater amount of memory than classical test generation algorithms, making their applicability not always feasible in practice. In this paper we propose several ways to optimize the SAT/SMT-based process of test generation for Boolean expressions and we compare several solving tools and propositional transformation rules. These optimizations promise to make SAT/SMT-based techniques as efficient as standard methods for testing purposes, especially when dealing with Boolean expressions, as proved by our experiments.
Automating Test Case Design within the Classification Tree Editor
"... Abstract—This paper describes how the proven test design technique of the classification tree method is extended within the classification tree editor in order to contribute to current test design matters. The classification tree editor not only provides the tooling to use the method but also to app ..."
Abstract
- Add to MetaCart
(Show Context)
Abstract—This paper describes how the proven test design technique of the classification tree method is extended within the classification tree editor in order to contribute to current test design matters. The classification tree editor not only provides the tooling to use the method but also to apply new and helpful features in the test design process. This includes the automatic generation of test cases and test sequences according to desired test depth and focus, automated boundary value analysis, various tool couplings to integrate in each individual test process and supporting features like test evaluation or test coverage analysis amongst others. Keywords-classification tree method; classification tree editor; combinatorial interaction testing; I.
