Recent years have seen the emergence of a new generation of heavily-optimised modal decision procedures. Several systems based on such procedures are now available and have proved to be much more effective than the previous generation of modal decision procedures. As both computational complexity and algorithm complexity are generally unchanged, neither is useful in analysing and comparing these new systems and their various optimisations. Instead, empirical testing has been widely used, both for comparison and as a tool for tuning systems and identifying their strengths and weaknesses. However, the very effectiveness of the new systems has revealed serious weaknesses in existing empirical test suites and methodologies. This paper provides a detailed survey of empirical testing methodologies, analyses the current state of the art and presents new results obtained with a recently developed test method.

Modal nonmonotonic logics constitute a well-known family of knowledge -representation formalisms capturing ideally rational agents reasoning about their own beliefs. Although these formalisms are extensively studied from a theoretical point of view, most of these approaches lack generally available solvers thus far. In this paper, we show how variants of Moore's autoepistemic logic can be axiomatised by means of quantified Boolean formulas (QBFs). More specifically, we provide polynomial reductions of the basic reasoning tasks associated with these logics into the evaluation problem of QBFs. Since there are now efficient QBF-solvers, this reduction technique yields a practicably relevant approach to build prototype reasoning systems for these formalisms. We incorporated our encodings within the system QUIP and tested their performance on a class of benchmark problems using different underlying QBF-solvers.

We provide an empirical evaluation of one of the main test sets that is currently in use for testing modal satisfiability solvers, viz. the random modal QBF test set. We first discuss some of the background underlying the test set, and then evaluate the test set using criteria set forth by Horrocks, Patel-Schneider, and Sebastiani. We also present some guidelines for the use of the test set.

. Previous methods for generating random modal formulae (for the multi-modal logic K (m) ) result either in flawed test sets or formulae that are too hard for current modal decision procedures and, also, unnatural. We present here a new system and generation methodology which results in unflawed test sets and more-natural formulae that are better suited for current decision procedures. Most empirical testing of decision procedures for propositional modal logics, usually for the multi-modal logic K (m) , employs randomly generated formulae. This style of testing was initially proposed by Giunchiglia et al [3] and later improved by them and also by Hustadt and Schmidt [6, 1]. Other kinds of randomly generated formulae have been proposed by Massacci [7]. Randomly generated formulae have been used with all the recent, highly optimised modal decision procedures, including DLP [8], FaCT [4], KSatC [1], *SAT [2], and TA [6], and have been used in several comparisons of these systems [7]...

. We present a data type |that we call \bit matrix"| for caching the (in)consistency of sets of formulas. Bit matrices have three distinguishing features: (i) they can query for subsets and supersets; (ii) they can be bounded in size; and (iii) if bounded, the latest obtained (in)consistency results can be kept. We have implemented a caching mechanism based on bit matrices in *sat. Experimenting with the TANCS 2000 benchmarks for modal logic K, we show that bit matrices (i) allow for considerable speedups, and (ii) lead to better performances both in space and time than the natural alternative, i.e., hash tables. 1 Introduction The implementation of ecient decision procedures for modal logics is a major research problem in automated deduction. Several systems have been developed, like Kris [1, 2], Ksat [3, 4], LWB [5], FaCT [6], and more recently *sat [7, 8], KK [9], dlp [10], HAM-ALC [11], KtSeqC [12], and MSPASS [13]. A competition is run in conjunction with the Tableaux co...

Abstract not found

In recent years, an important number of theoretical results concerning axiomatizability, proof systems (tableaux, natural deduction, etc.), interpolation, expressive power, complexity, etc. for hybrid logics has been obtained. The next natural step is to develop provers that can handle these languages. HyLoRes is a direct resolution prover for hybrid logics implementing a sound and complete algorithm for satisfiability of sentences in H (@;#). The most interesting distinguishing feature of HyLoRes is that it is not based on tableau algorithms but on (direct) resolution.

Introduction Hybrid languages are modal languages that allow direct reference to the elements of the model. Already the basic hybrid language (H(@)) which extends the basic modal language with the addition of nominals (i; j; k; : : :) and satis ability operators (@ i ; @ j ; @ k ; : : :), obviously increases the expressive power of the language as we can now explicitly check whether the actual point of evaluation is some speci c, named point in the model (w i), and whether a named point satis es a given formula (w @ i '). More interestingly, this extended expressive power permits also the de nition of very elegant decision algorithm, where nominals and @ play, inside the object language, the role of labels, or pre xes, which are usually needed during the construction of a proof in the modal setup (see, e.g. [8, 4]). And all these features we get with no increase in complexity (up to a polynomial): the complexity of the satis ability problem for H(@) is the same as for the ba

We report on ongoing experimental work on evaluating test sets for testing modal satisfiability solvers. Our longterm aim is to understand the difference between the use of structured and of unstructured randomly generated problems. Which parts of the problem space do they explore?

GridTest is a framework for testing automated theorem provers using randomly generated formulas. It can be used to run tests locally, in a single computer, or in a computer grid. It automatically generates a report as a PostScript file which, among others, includes graphs for time comparison. We have found GridTest extremely useful for testing and comparing the performance of different automated theorem provers (for hybrid, modal, first order and description logics). We present GridTest in this framework description in the hope that it might be useful for the general community working in automated deduction.