C. B. Suttner and G. Sutclie. The TPTP problem library (TPTP v2.2.0). Tech. Report 99/02, James Cook University, Townsville, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....the set S of all successors [11] thereby re ning the functionality speci ed in Line 10 of Alg. 1. The conjecture u=v is proved if S(u) S(v) 6= Our method is best explained with a simple example. Figure 2 shows how sets of successors can be used in solving the proof problem GRP141 1 [27]. The terms u f(f(a; b) c) and v c have to be joined. The four graphs illustrate the enlarging of the conjecture. Initially, the two sets consist of only one term each (a) With the 16th rewrite rule u can be reduced to two di erent terms which join again in the term denoted by u . S(u) ....

C. B. Suttner and G. Sutclie. The TPTP problem library (TPTP v2.2.0). Technical Report 99/02, James Cook University, Townsville, 1999.

....systems or other intelligent systems. It is well known that libraries or corpus of problems used to evaluate systems in an area tend to produce specialised systems which attain better scores on that kind of problems while being completely dull at others. This has been the case even in TPTP [Suttner Sutcliffe 1996], a complete and varied fair corpus used to compare Automatic Reasoning systems or ATP devices. The idea of making feasible more objective measures for intelligent systems is not new. There are prediction contests based on chaotic strings [Ditto Munakata 1995] We hope that if our C tests are ....

Suttner, C.B.; Sutchliffe, G. "The TPTP Problem Library", Tech. Univ. Munich, Germany, 1996

....are thrown away, and just minimal reconstruction information is kept, at constant cost. The in uence of these representations on the process size over abstract time can be studied in Fig. 1. For each variant, the Waldmeister system was run on the ROB001 1 proof task from the TPTP problem library [SS99] using an UltraSPARC IIi workstation. The process size minus the space consumption of P serves as a baseline (lowest curve) Stringterms allow compression rates of about 10, which is comparable to what can be achieved with a shared term representation [LS01] The overlap representation improves on ....

....overlap representation is apparently the best one in terms of space. But contrary to one s expectations it may in uence the behavior of the proof search. To quantify the e ect, we performed the following experiment: On the 431 unit equality proof problems in the current TPTP release [SS99] Waldmeister is run with six di erent settings, two orderings (KBO LPO) combined with three general purpose weighting functions (addweight mixweight gtweight) For each of the 2586 cases we compare the prover s behavior using stringterms with that using overlap representation. Within the resource ....

[Article contains additional citation context not shown here]

C. B. Suttner and G. Sutclie. The TPTP problem library (TPTP v2.2.0). Technical Report 99/02, James Cook University, Townsville, 1999.

....is there to show what happens when the query does not fail. It uses the list representation and also omits the argument collecting the plan. Finally, BOO019 1 is an axiomatisation of a ternary boolean algebra, a typical problem from the theorem proving community taken from the TPTP library [29]) Its only predicate is equality, whose interpretation can be fixed to the identity (so the number of different interpretations is only 1 instead of 2 9 ) The abductive system AB uses the control as described in Section 4.1 and is augmented with the intelligent backtracking and symmetry ....

C.B. Suttner and G. Sutcliffe. The TPTP problem library, version 2.1.0, 15/12/1997. Technical Report 97/08, James Cook University, Australia, 1997.

....[prime divisor ## X]X]# #X[prime X # primeprime divisor X] ##X[prime X # less [prime divisor X]X] # prime a # ##X#prime X##less a X # less [factorial plus one a] X (search: 7.38 sec merge: 5.73 sec translate: 6. 23 sec total: 20.79 sec) This is example NUM016 1 from the TPTP Problem Library [54]. It is more commonly referred to as LS17. SYN031 1: ##A # [g o## Aa # # g[f ## A]A] ##A[gAa # gA fA]##A#B # [#gAB # g[fB]B] # #A #B[#gAB # gB fB]##A#B#gAB ##gBa (search: 64.08 sec total: 65.79 sec) This is example SYN031 1 from the TPTP Problem Library. It is more commonly referred to as ....

Sutcliffe, G., Suttner, Ch., and Yemenis, T.: The TPTP problem library, in A. Bundy (ed.), Automated Deduction -- CADE-12, 12th Int. Conf. Automated Deduction, Nancy, France, Lecture Notes in Artificial Intelligence 814, Springer-Verlag, 1994, pp. 252--266.

....from inside KIV, whereas for the other provers the problems had to be transferred by hand. As a general result of the experiments we found that the automated provers did not satisfy the expectations induced by the good results some of them normally get in standard benchmarks from the TPTP library [13]. In our experiments either the success rate was rather low or the required time was unexpectedly high. One major reason for this behavior is the large number of axioms in typical software speci cations. Whereas typical TPTP problems only have a few axioms, typical software speci cations in KIV ....

G. Sutclie, C. Suttner, and T. Yemenis. The TPTP problem library. In A. Bundy, editor, 12th International Conference on Automated Deduction, CADE12, Springer LNCS 814. Nancy, France, 1994.

....between inputs rather than simply setting a fixed value for 71 any input. 5.1 Input Problems Sttrim was run on two major problem sets, plus a handful of problems that were generated during its construction. One of the problem sets was the propositional problems from the SYN category of TPTP [50]. The other major set of problems came from a test pattern generator [39] for particular kinds of hardware faults. Finally, a few circuit implementations and specifications were coded here at the University of North Carolina during the early construction of the prover, for proof of concept. ....

....kinds of hardware faults. Finally, a few circuit implementations and specifications were coded here at the University of North Carolina during the early construction of the prover, for proof of concept. 5.1. 1 TPTP The TPTP problem library contains both propositional and first order problems [50]. Sttrim, of course, is a propositional theorem prover. However, first order problems which are ground can be treated as propositional. Some of the problems in the TPTP library have generators , which allow for the same basic problem to be instantiated with multiple sizes. Note that the TPTP ....

G. Suttcliffe and C. B. Suttner. The TPTP problem library. Technical Report 96/09, James Cook University, Australia, 1996.

....and removing redundant clauses. Resolution based provers are currently evaluated on empirical grounds. Comparisons between provers are mostly based on success rates and run times on standard corpora of problems. The main corpus is the TPTP (Thousands of Problems for Theorem Provers) library, Sutcli#e et al. 1994 ] This is also used as the basis for an annual competition between provers run in conjunction with the CADE conferences, Suttner and Sutcli#e, 1998 ] There is also a lot of interest in tackling open conjectures in mathematics. Conjectures best suited to this approach are combinatorial ....

G. Sutcli#e, C. Suttner, and T. Yemenis. The TPTP problem library. In Bundy [ 1994 ] , pages 252--266.

....do not formally prove that a sort is in fact freely generated but just generate the additional axioms. Although these axioms do not capture the finite generation property, this approximation works quite well in practice. 3. 3 Simplification Unlike the problems in benchmark collections as the TPTP [SSY94], proof tasks in applications are generated automatically and thus not simplified. e.g. in our case they may still contain the propositional constants true and false from the original contracts or redundant equations which may be used to simplify the task. Hence, rigorous simplification is a ....

G. Sutcliffe, C. B. Suttner, and T. Yemenis. "The TPTP Problem Library". In A. Bundy, (ed.), Proc. 12th Intl. Conf. Automated Deduction, Lect. Notes Artifical Intelligence 814, pp. 252--266, Nancy, June-July 1994. Springer.

....The equational reasoning system employed for our experiments was the Discount system [3] which is based on the unfailing Knuth Bendix completion procedure (e.g. 5] 1 The problems were taken from the public TPTP ( Thousands of Problems for Theorem Provers ) problem library version 1.2. 1 [12]. More precisely, we selected a total of 263 problems from six problem classes BOO, COL, GRP, LCL, RNG, and ROB. These acronyms are used and explained by the TPTP. 1 For readers interested in technical details: for all experiments Discount employed the reduction ordering LPO with a precedence ....

G. Sutcliffe, C. Suttner, and T. Yemenis. The TPTP problem library. In Proc. 12th Conference on Automated Deduction (CADE-12), LNAI 814, pages 252--266. Springer, 1994.

....54s 58s 9s CAT018 3 74s LDA011 2 21s 35s 7s CAT019 4 of the ME prover. Thus, we can achieve cooperation by exchanging lemmas and subgoal clauses without one concept disturbing the other. We experimented in the light of problems from the well known problem library TPTP [SSY94]. In order to obtain a reliable collection of data, we employed two domains of TPTP as our test set, the domains CAT (category theory) and LDA (LD algebras) The CAT domain consists of 58 problems, the LDA domain of 22. From these domains we extracted 22 and 15 non trivial problems, respectively, ....

....Letz, J. Steinbach, C. Goller, J. Schumann, and K. Mayr. The Model Elimination Provers SETHEO and E SETHEO. Journal of Automated Reasoning, 18(2) 1997. Sch94] J. Schumann. Delta a bottom up preprocessor for top down theorem provers. system abstract. In Proceedings of CADE 12. Springer, 1994. [SSY94] G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In CADE 12, pages 252 266, Nancy, 1994. LNAI 814. Sti88] M.E. Stickel. A prolog technology theorem prover: Implementation by an extended prolog compiler. Journal of Automated Reasoning, 4:353 380, 1988. Sut92] G. ....

[Article contains additional citation context not shown here]

G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In Proc. of the 12th CADE, Nancy, number 814 in LNAI, pages 252--266. Springer, 1994.

....depth requirement, that is d min = 0. Table 1 shows the results obtained with this setting for five problems of CD. The problem names given in the first column are the names used by the TPTP (Thousands of Problems for Theorem Provers) a large public collection of problems for theorem provers [Sutcliffe et al. 1994]. The second and third column show the sizes of POS and NEG for the respective problem. The heuristic used by CoDe to produce POS 5 Table 1: Results (fitness of best individual) of the GA and random search (RND) for five problems of CD. Problem jPOS j jNEG j GA RND LCL006 1 8 540 0.0695 ....

Sutcliffe, G., Suttner, C., and Yemenis, T. (1994). The TPTP problem library. In Proc. 12th Conference on Automated Deduction (CADE-12), LNAI 814, pages 252--266. Springer. See also http://www.cs.jcu.edu.au/~tptp.

....relatively small. Nonetheless, it has to deal with large amounts of intermediate results (critical pairs, for example) These intermediate facts will usually be simplified extensively. y Many of the original problems specified by the ILF group have been incorporated into the TPTP problem library (Sutcliffe et al. 1994)) Jorg Denzinger, Stephan Schulz Table 1. Descriptions and run times for some problems Problem Team Best Exp. Description Lusk2 N A 0.033 Prove that a monoid with x 2 = x is Abelian. This was presented as a very basic example in Lusk and Overbeek (1982) A proof protocol for this ....

Sutcliffe, G., Suttner, C.B., Yemenis, T. (1994). The TPTP Problem Library. In Proc. of the 12th CADE, Nancy, number 814 in LNAI, pages 252--266. Springer.

....the hardest formula with the LWB. This situation is very unsatisfactory. Therefore we decided to collect a set of benchmark problems for the propositional modal logics K, KT, and S4. In classical predicate logic, the often cited collection of Pelletier [9] has been replaced by the TPTP library [11]. Although this library is considerably large, it is still common to choose a dozen of these formulas out of this library and publish the execution times for these formulas. Therefore we do not just give a list of formulas, but present a method that makes it possible to compare di erent provers. ....

....give a list of formulas, but present a method that makes it possible to compare di erent provers. The producers of TPTP plan a so called benchmark suite for the next version that should allow the computation of a performance index for automated theorem provers for classical predicate logic; cp. [11]. Work supported by the Swiss National Science Foundation, 21 43197.95. 1 The selection of the hard benchmark formulas was guided by the following postulates (see the following section for a thorough discussion) 1. Provable as well as unprovable formulas. 2. Formulas of various structures. ....

G. Sutclie, C. Suttner, and T. Yemenis. The TPTP problem library. In A. Bundy, editor, CADE 12, LNCS 814, pages 252-266, 1994. 21

....in the sense that it uses natural language exclusively, it is an axiomatic theory in which axioms and theorems are clearly outlined. 4 This greatly facilitates 3 Some fragments of the resulting formalizations are included in the TPTP (Thousands of Problems for Theorem Provers) Problem Library (Sutcliffe et al. 1994). 4 Note that this is an exceptional case; only few social scientists state carefully formulated propositions, and even fewer authors attempt to make their underlying assumptions explicit. It is also possible to reconstruct less explicit the rational reconstruction of the theory, and allows us ....

Sutcliffe, G., C. Suttner, and T. Yemenis (1994). The TPTP problem library. See Bundy (1994), pp. 252--266.

....they need to be adapted in order to be useful for the intended application. This is due to the fact that most ATPs have been designed and evaluated with the aim to solve small, but hard problems as they typically occur in mathematics. The large collection of problems in the TPTP benchmark library (Sutcliffe et al. 1994) reflects this fact: most problems are specifically formalized and prepared for automated deduction. Proof tasks from applications in the area of high quality software design usually poses quite different requirements for the automated theorem prover (cf. Schumann, 1999a) Extensions needed to ....

Sutcliffe, G., Suttner, C., and Yemenis, T. (1994). The TPTP Problem Library. In Proc. 12th International Conference on Automated Deduction (CADE 12), volume 814 of LNAI, pages 252--266. Springer.

....first order theorems that showed up during proofs of specification and program properties in KIV. The five provers were Otter ( 11] Protein ( 1] Setheo ( 3] Spass ( 9] and 3 T A P ( 2] The challenge for the provers in this application domain (unlike in standard TPTP benchmarks, [8]) is the large number of up to several hundred axioms in typical software specifications. We found that both the success rates and the proof times of the automated provers strongly depend on how good they are able to find out the few relevant axioms that are really needed in the proofs. In this ....

G. Sutcliffe, C. Suttner, and T. Yemenis. The tptp problem library. In A. Bundy, editor, 12th International Conference on Automated Deduction, CADE-12, Springer LNCS 814. Nancy, France, 1994.

....to their effect on the set of support. At present, the program does not do this much meta reasoning, though we hope to include it in a future release. 3 Experimental Results The development of MScott has been underpinned by extensive experimentation within the TPTP v2.0. 0 problem library [6]. This was motivated by the disappointing performance of its predecessor Scott in the CASC 14 systems competition [5] Here Scott was outperformed by its base program Otter, arousing concern about the effectiveness of the strategy. Comparative tests of Scott and the equivalent version of Otter ....

G Sutcliffe, C Suttner and T Yemenis. The TPTP Problem Library. Proceedings of 12th International Conference on Automated Deduction (1994), 252--266. This article was typeset using the L A T E X macro package with the LLNCS2E class.

....seems promising, in particular for satis able or non Horn problems without equality (there is no built in equality treatment yet) In the respective subdivisions SAT and NNE of the CASC 16 system competition 1999, FDPLL scored rank 4 of 6 and rank 4 of 10, respectively. From the TPTP library [SSY94], FDPLL can also solve some dicult unsatis able problems quite quickly (e.g. ANA002 4, the intermediate value theorem, in 3 seconds) The overall success rate is about 40 (Otter: 52 ) for a time limit of 10 minutes. Other sources for future work are combinations of the techniques described here ....

G. Sutclie, C. Suttner, and T. Yemenis. The TPTP problem library. In Alan Bundy, editor, CADE 12, LNAI 814, pp. 192-206, Nancy, France, June 1994. Springer.

....the hardest formula with the LWB. This situation is very unsatisfactory. Therefore we decided to collect a set of benchmark problems for the propositional modal logics K, KT, and S4. In classical predicate logic, the often cited collection of Pelletier [10] has been replaced by the TPTP library [12]. Although this library is considerably large, it is still common to choose a dozen of these formulas out of this library and publish the execution times for these formulas. Therefore we do not just give a list of formulas, but present a method that makes it possible to compare di#erent provers. ....

....a list of formulas, but present a method that makes it possible to compare di#erent provers. Also the producers of TPTP plan a so called benchmark suite for the next version that should allow the computation of a performance index for automated theorem provers for classical predicate logic; cp. [12]. # Work supported by the Swiss National Science Foundation, 21 43197.95. 1 The selection of the hard benchmark formulas was guided by the following postulates: 1. Provable as well as not provable formulas. 2. Formulas of various structures. 3. Some of the benchmark formulas are hard enough ....

G. Sutcli#e, C. Suttner, and T. Yemenis. The TPTP problem library. In A. Bundy, editor, CADE 12, LNCS 814, pages 252--266, 1994. 17

....and removing redundant clauses. Resolution based provers are currently evaluated on empirical grounds. Comparisons between provers are mostly based on success rates and run times on standard corpora of problems. The main corpus is the TPTP (Thousands of Problems for Theorem Provers) library, Sutcliffe et al. 1994 ] This is also used as the basis for an annual competition between provers run in conjunction with the CADE conferences, Suttner and Sutcliffe, 1998 ] There is also a lot of interest in tackling open conjectures in mathematics. Conjectures best suited to this approach are combinatorial ....

G. Sutcliffe, C. Suttner, and T. Yemenis. The TPTP problem library. In Bundy [ 1994 ] , pages 252--266.

....called from inside KIV, whereas for the other provers the problems had to be transferred by hand. As a general result in the experiments we found that the automated provers did not satisfy the expectations induced by the good results some of them normally get in standard benchmarks from the TPTP ([SSY94]) library. In our experiments either the success rate was rather low or the required time was unexpectedly high. This research was sponsored by the German Research Foundation (DFG) under grant Re 828 2 2. 2 between the research groups of 3 T A P and KIV at the Universities of Karlsruhe and ....

G. Sutcliffe, C. Suttner, and T. Yemenis. The tptp problem library. In A. Bundy, editor, 12th International Conference on Automated Deduction, CADE-12, Springer LNCS 814. Nancy, France, 1994.

....and Results In this section we look in detail at the results, obtained with the three different versions of SiCoTHEO. The experiments have been carried out on a network of HP 750 workstations, connected via Ethernet. All formulae for the experiments have been taken from the TPTPproblem library [12]. 6 5.1 SiCoTHEO PID Parallel iterative deepening is one of the simplest forms of competition: each processor explores the search space to a specific bound. For example, if we have the A literal depth as the bound, we might want to start processor number i with depth bound i. In order to ensure ....

....has not been tested for larger networks of processors (and possibly different ranges for ff and fi) seq. run time [ # samples] P = 9 P = 25 P = 50 1s T seq 10s[3] s a 3.59 3.58 15.32 s g 2.00 2.00 3.77 s h 1.35 1.35 1.64 s t 2.40 2.40 3. 25 10s T seq 100s[12] s a 37.89 39.33 51.81 s g 18.80 20.64 25.26 s h 5.65 7.43 7.68 s t 4.81 6.80 7.00 100s T seq [36] s a 65.40 74.52 110.10 s g 9.65 14.40 16.14 s h 2.86 3.56 3.72 s t 2.66 3.34 3.51 Table 2: SiCoTHEo CBC: Mean values of speed up for ....

G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In CADE 12, Springer 1994.

....a is 2#(1 0.5#1 4) 1.75. As there is no TSA corresponding to the last node containing the variable X , we assign a weight of 1, giving a total weight of 4.25 for the complete term. Experimental Results We used the set of all unit equality problems from the TPTP problem library, version 2.1. 0 (Suttner Sutcli #e 1997), as a test case. The set of training examples contains 201 proofs for problems that could be found by DISCOUNT within a 180 second time limit using DISCOUNT s best conventional strategy, AddWeight 4 . Table 1 shows results for AddWeight, for Occnest (a goal directed strategy) and 4 di#erent ....

Suttner, C., and Sutcli#e, G. 1997. The TPTP Problem Library (TPTP v2.1.0). Technical Report AR-9701 (TUM), 97/04 (JCU), Institut fur Informatik, TU Munchen, Munich, Germany/Department of Computer Science, James Cook University, Townsville, Australia.

.... problem we want to take a closer look at is a problem of the equivalential calculus, namely A 1 = Ax 1 ; 1 ) with Ax 1 = fe(e(x; y) e(y; x) e(e(e(x; y) z) e(x; e(y; z) g and 1 j e(e(e(x; y) e(z; x) e(y; z) Theorem 5 in [17] Problem 69 in [9] or problem LCL006 1 in the TPTP [14]) In [17] a first proof obtained with Otter has length 33. Please note that [17] measures proof length in terms of proof steps which is equivalent to the number of derived facts excluding axioms. Hence, our notion of proof length is equal to the number of proof steps in [17] plus the number ....

.... y) z) 11 [ 7 , 2 ] e(e(e(e(z; y) x) z) e(y; x) 12 [ 7 , 8 ] e(e(z; y) e(e(e(y; x) x) z) 13 [ 7 , 10 ] e(e(z; e(y; x) e(e(x; y) z) 14 [ 11 , 9 ] e(z; e(y; e(x; e(e(y; x) z) 15 [ 1 , 12 ] e(e(e(e(z; y) y) x) e(x; z) 16 [ 13 , 9 ] e(e(z; y) e(x; e(y; e(z; x) 17 [ 7 , 14 ] e(e(z; y) e(x; e(e(y; x) z) 18 [ 7 , 16 ] e(e(e(z; y) x) e(y; e(z; x) 19 [ 15 , 17 ] e(e(z; e(e(y; z) e(x; y) x) 20 [ 18 , 19 ] e(e(e(z; y) e(x; z) e(y; x) The maximal size of a fact in this proof is 11 (the same as in Otter s proofs) When using H 1;0;0 (pure depth based ....

[Article contains additional citation context not shown here]

G. Sutcliffe, C. Suttner, and T. Yemenis. The TPTP problem library. In Proc. 12th Conference on Automated Deduction (CADE-12), LNAI 814, pages 252--266. Springer, 1994. 14

....Table 1 shows the performance of the two standard heuristics described above and the TSM based heuristic in terms of problems solved in a given (perproblem) time limit. We used the set of all unit equality problems with universally quantified goals from version 2.0. 0 of the TPTP problem library [SSY94], i.e. all problems suitable for using ordinary unfailing completion. This set consist of 303 problems, 247 for which there are known proofs, 51 which should be provable, but no known proof exists, and 5 of entirely open status. Tests were performed on a SUN SPARCstation Ultra 2 200, all times ....

G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In Proc. of the 12th CADE, Nancy, number 814 in LNAI, pages 252--266. Springer, 1994.

....for rigid hypertableaux as it is the case for other proof confluent tableau calculi with exception of the disconnection calculus by Billon (1996) 3. 3 Experimental results I have tested my hypertableaux therorem prover which is written in ECLiPSe Prolog with the 2759 problems of the TPTP library (Sutcliffe et al. 1994) on a SPARC 10 restricting the runtime to 90 seconds. In this time, 513 problems have been solved, where for 303 problems runtime has been below 1 second and for 114 problems below 10 seconds. In contrast to other automated theorem provers, Hyper, hyper detected 29 from the 59 satisfiable ....

....tableau calculi in section 3 and offered a possible solution by term depth iterating together with a redundancy check based upon rigid literal subsumption. With this fairness criterion a proof procedure for rigid hypertableaux has been implemented and evaluated with the TPTP problem library (Sutcliffe et al. 1994). Although the implementation is non optimized Prolog code, there are encouraging experimental results, especially concerning satisfiability detection and memory consumption. Remaining task is to finish the model construction approach mentioned in section 2 and to establish theoretically the ....

G. Sutcliffe, C. Suttner, and T. Yemenis. The TPTP problem library. In A. Bundy, editor, Proc. CADE-12, volume 814 of LNAI. Springer, 1994.

....set004 OEL 12 5 6.12 0.05 none 48 8 0.72 set005 any 500 300 set006 OEL 650 40 52.15 0.05 none 500 set007 any 500 300 set008 OEL 1406 536 102.29 0.96 none 500 set009 OEL 22 10 0.27 0.47 none 22 10 0.17 Fig. 3. Results for TPTP problem SET a few problems from the TPTP problem class SET [19]. The calculus used are tableaux with input selection function. None of the problems is formulated with equality. As the time needed for refutation is only a rough measure of the complexity of a proof, we give the number of rule applications (R) 4 and of branches (CB) in the closed tableau. ....

G. Sutcliffe, C. Suttner, and T. Yemenis. The TPTP problem library. In A. Bundy, editor, Proc. 12th Conf. on Automated Deduction CADE, Nancy, LNAI, pages 252--266. Springer, 1994. This article was processed using the L A T E X macro package with LLNCS style

....problems. Although some hard problems can be solved using a trivial semantics, the prover performs best when a natural semantics is used. We generated natural semantics for many problems by hand. The process is easier than it sounds. For example, many of the GRP problems from TPTP library (Sutcliffe, Suttner, Yemenis 1993) have the same set of axioms. Had the library used a uniformed set of naming scheme for constant symbols (not just functions and predicates) a single semantics would have sufficed for all the GRP problems We hope that the TPTP library will eventually include semantics for the appropriate ....

Sutcliffe, G.; Suttner, C.; and Yemenis, T. 1993. The TPTP problem library. Technical Report 93/11, Department of Computer Science, James Cook University, Australia.

....examples of application of the method described above. Both of them have been mechanically treated by an early implementation of the generalization rules. The computing time on a SUN 4 ELC is less than 1 second for both the following examples. The first one is problem MSC003 1 of the tptp library [SS95] 4 . It deals with numerical computation of members with respect to class belonging. We will see that the actual values of numbers can be eliminated of the generalization, therefore doing it possible to solve the problem with different values. i (o (A, B, C, D, E) B) p (A, t (D, E) C) ....

....and Perspectives A method for analogy based theorem proving or counterexample building has been presented. One of the main reasons for developing it on a resolution ground was that this choice makes it adequate to be used on very large databases of theorems (and non theorems) such as tptp [SS95]. The main goals of the work are: a unified approach to the discovery of proofs or building of counterexamples by analogy; a method to generalize proofs (or model constructions) Further research will be lead about conditions to apply to the rewriting rules presented in this work, so that ....

Ch. B. Suttner and G. Sutcliffe. The TPTP problem library. Technical report, TU Munchen / James Cook University, 1995. V-1.2.0.

....to evaluate our method in the light of experiments with the model elimination prover Setheo and the lemma generator Delta. We conducted experiments in the area of PL1 (with equality) In detail we have performed experiments in the BOO, CAT, COL, and GRP domain of the public problem library TPTP [SSY94] There have been two main aims by performing these experiments. On the one hand we want to demonstrate that similarity based lemma generation indeed improves on an unfiltered use of lemmas, a filtering which neglects the proof task at hand, and a version of Setheo which does not employ lemmas. ....

G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In Proceedings of the 12. International Conference on Automated Deduction (CADE), pages 252--266. Springer LNAI 814, 1994.

....different times to find a proof, Functional Satchmo solved only a proper subset of the problems Compiling Satchmo solved. 4.2. Experimental Results Outside the Competition In a comparison [15] of Compiling Satchmo with MGTP G [8] Otter [12] and SETHEO [6] using the TPTP Problem Library [17] as a benchmark, Compiling Satchmo has turned out to be the most efficient for range restricted problems and for non nesting problems, i.e. precisely those for which Satchmo had been designed. Experimental results with Functional Satchmo after the competition show a significant speedup with a ....

G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In [2], pages 252--266.

....of Robbins algebra and group theory. We report on experiments performed with the problems available in the ROB and GRP domain 3 Note that the standardization allows us to explicitly consider a left and a right hand side of an equation when computing feature values. 15 of the TPTP library ([16]) v.1.2.1. More exactly we tried to solve the problems in both domains that are not solvable with conventional heuristics of Discount within the time limit of 10 minutes using a SPARCstation 10 ( hard problems ) A comparison with the renowned Otter prover [13] demonstrates the quality of our ....

Sutcliffe, G.; Suttner, C.B.; Yemenis, T.: The TPTP Problem Library, Proc. CADE 12, Nancy, Springer LNAI 814, pp. 252-266,

....rules. The computing time on a SUN 4 ELC is less than 1 second for both the following examples. Those examples are just for clarity purpose. Our conditional rewrite system can obviously be applied to any derivation, of whatever size. The first one is problem MSC003 1 of the tptp library [20] 4 . It deals with numerical computation of members with respect to class belonging. We will see that the actual values of numbers can be eliminated of the generalization, therefore doing it possible to solve the problem with different values. i (o (A, B, C, D, E) B) p (A, t (D, E) C) ....

....and Perspectives A method for analogy based theorem proving or counterexample building has been presented. One of the main reasons for developing it on a resolution ground was that this choice makes it adequate to be used on very large databases of theorems (and non theorems) such as tptp [20]. The main goals of the work are: Gamma a unified approach to the discovery of proofs or building of counterexamples by analogy; article.tex; 19 09 1997; 15:16; no v. p.19 20 Gilles D efourneaux et al. Gamma a method to generalize proofs (refutations) or counterexamples (model ....

Ch. B. Suttner and G. Sutcliffe. The TPTP problem library. Technical report, TU Munchen / James Cook University, 1995. V-1.2.0.

....introduced by definitional translations. In this paper, we use a third criterion for the comparison, namely the run time (of our theorem prover KoM eT) to find proofs for the different normal forms of the same first order formula. An ff release of a first order formula version of the TPTP library [25] is used for the comparison. We demonstrate that definitional translations can be efficiently applied if reductions are used in the preprocessing phase to remove some apparently useless definitions. It turns out that approximately the same number of examples are solved applying the traditional ....

G. Sutcliffe, Ch. Suttner, and T. Yemenis. The TPTP Problem Library. In [8], pages 252--266.

....0 ) r(C;C 0 ) jC 0 j ; otherwise Otherwise, if C is not a unit clause, we define sp (C; C 0 ) 0 ; C 0 is not a unit clause r(C;C 0 ) jCj ; otherwise 4. 2 Experimental results We conducted experimental studies in the light of problems taken from the problem library TPTP ([SSY94]) As our test set we chose problems from the CADE 13 ATP system competition ( SS97] and selected those from the categories unit equality and mixed . Table 1 presents an excerpt of our results obtained when tackling hard and medium problems (the runtime is at least 8 seconds on a SPARCstation ....

G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In CADE-12, pages 252--266, Nancy, 1994. LNAI 814. REFERENCES 21

....but in putting the selected c clauses in the tail of the list instead of its head. A bottom up approach can be used as well: it is similar but begins on the output set of the derivation. 4 Example The following example, which we use to illustrate our method, uses theorems taken from the TPTP [14] library. We tried to focus on domains where theorem statements make it easier for the human to understand why analogy can be used. We choose problems from the groups domain. The source derivation is a refutation of problem GRP009 1 automatically done by OTTER [11] standard flags) The target set ....

C. Suttner and G. Sutcliffe. The TPTP problem library. Technical report, TU Munchen / James Cook University, 1996. V-1.2.1.

....: C n , one can apply this system of rules on C 1 ; C n in order to find the right instances of the c clauses. 5. Examples The two examples below show evidence of the interest of our approach. Example 5.1. Consider the following example, taken from the TPTP library, problem SYN324 [33]. 1 [ P (x; g(x) P (x; x) 2 [ P (x; g(x) P (x; x) 3 [ P (x; g(x) P (g(x) g(x) 4 [ P (x; g(x) P (g(x) g(x) Our method gives 5 [ P (x; x) P (g(x) g(x) resolution,1,4) 6 [ P (x; x) P (g(x) g(g(x) resolution,5,2) 7 [ P (x; x) P ....

C. B. SUTTNER and G. SUTCLIFFE. The TPTP problem library. Technical report, TU Munchen / James Cook University, 1995. V-1.2.0.

....is not complete. We measured the number of clauses built by OTTER that were not tautologies, and the number of binary resolution trees built by the prototype that were minimal. The results are incomplete, but encouraging. For instance, OTTER accepts 10091 clauses to refute SYN094 1. 005 from TPTP (Sutcliffe, Suttner, Yemenis 1994), whereas the prototype allows only 359 minimal binary resolution trees. OTTER needs 35820 inferences for the four pigeons problem, MSC007 1.004, while the prototype needs 577. In a minority of our experiments OTTER needed fewer inferences because the more restricted search space of the prototype ....

Sutcliffe, G.; Suttner, C.; and Yemenis, T. 1994. The TPTP problem library. In Kapur, D., ed., Automated Deduction CADE-12, number 814 in Lecture Notes in Artificial Intelligence, 252--266. Springer-Verlag, Berlin.

....to add some clauses that the bottom up prover has generated to the axiomatization of the ME prover. Thus, we can achieve cooperation by exchanging lemmas and subgoal clauses without one concept disturbing the other. We experimented in the light of problems from the well known problem library TPTP ([SSY94]) In order to obtain a reliable collection of data, we employed two domains of TPTP as our test set, the domains CAT (category theory) and LDA (LD algebras) The CAT domain consists of 58 problems, the LDA domain of 22. From these domains we extracted 22 and 15 hard problems, respectively, i.e. ....

G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In CADE-12, pages 252--266, Nancy, 1994. LNAI 814.

....extracting the data. This holds doubly for reusing a colleague s data. So, in the past, almost all experiments had to be repeated to answer a new question, even if similar runs had already been done. If you consider that testing a small set of parameters on a problem library like the TPTP [9] means a sleepless night for up to 100 processors, it is worth recycling the old and useful, but often chaotic and (previously) difficult to access, data. Another consideration is an ATP system installed outside the developing group: it is difficult to obtain feedback and data on the tasks (not) ....

....development, testing and application specific tuning of automated theorem provers. The success of SETHEO at the CADE 13 system competition [10] shows the promise of a tool that allows more efficient tuning and testing of a prover against an extensive library of proof tasks such as the TPTP[9] library. The framework is available on the World Wide Web[4] ....

Sutcliffe G., Suttner C. B., Yemenis T.: The TPTP Problem Library. Proc. CADE-12, 1994.

....originally used algorithm GTP inference rights employing the techniques presented in section 4. We applied CoDe to proof problems stemming from the area of logic calculi which McCune and Wos tackled in [8] with their renowned prover Otter. These problems can also be found in the TPTP library ([13]) version 1.2.1, namely in the LCL domain. In order to show that the speed ups we achieved with our techniques are not due to the weakness of our basic prover we compare the results obtained with CoDe with those of Otter. The results with CoDe were obtained on a SPARCstation ELC, the results with ....

Sutcliffe, G.; Suttner, C.; Yemenis, T.: The TPTP Problem Library, Proc. CADE-12, Nancy, FRA, 1994, LNAI 814, pp. 252--266.

....only have to prove the lemma: 9y:l(g(y) i.e. to prove that S T [f:l(g(y) g is not satisfiable. Any theorem prover can do that. 6 Examples In this section we give two more realistic examples. Example 6 (Unsatisfiable set of clauses) Let us consider the problems Syn310 1 and Syn312 1 of Tptp [Suttner and Sutcliffe, 1996]. They belong to the class Pvd [Ferm uller et al. 1993] Problem SS is: p(x2; x1; x) p(x; x1; x2) p(x1; x; x2) p(x; x1; x2) p(x; x1; g(x2) p(x; x1; x2) p(f(x) x1; x2) p(x; x1; x2) p(a; b; c) p(f(g(a) f(g(b) f(g(c) S T is: p 0 (x; x3; x2) p 0 (x; x1; x2) p 0 (x1; ....

Ch. Suttner and G. Sutcliffe. The TPTP problem library. Technical report, TU Munchen / James Cook University, 1996. V-1.2.1.

....SPASS 0. 45 : 19 4 The Format of the Results 20 5 Conclusion 21 6 The ATP System Results 25 1 Introduction The TPTP (Thousands of Problems for Theorem Provers) Problem Library [SSY94, SS95] is a library of test problems for automated theorem proving (ATP) systems, using the clause normal form of 1st order logic. Since its first release in 1993, many researchers have used the TPTP as an appropriate and convenient basis for ATP system evaluation. Some researchers, who have tested ....

G. Sutcliffe, C.B. Suttner, and T. Yemenis. The TPTP Problem Library. In A. Bundy, editor, Proceedings of the 12th International Conference on Automated Deduction, number 814 in Lecture Notes in Artificial Intelligence, pages 252--266. Springer-Verlag, 1994.

No context found.

C. B. Suttner and G. Sutclie. The TPTP problem library (TPTP v2.2.0). Tech. Report 99/02, James Cook University, Townsville, 1999.

No context found.

G. Sutcliffe, C. Suttner, and T. Yemenis. The TPTP problem library. In Proc. CADE-12. Springer, 1994.

No context found.

G. Sutcliffe, Ch. Suttner, and T. Yemenis. The TPTP problem library. In A. Bundy, editor, Proceedings of the Conference on Automated Deduction, volume 814 of Lecture Notes in Artificial Intelligence, pages 252--266. Springer Verlag, 1994.

No context found.

C. B. Suttner and G. Sutcliffe. The TPTP Problem Library. Technical Report TR 95/6, James Cook University, Australia, August 1995.

No context found.

Sutcliffe, G.; Suttner, C.B.; Yemenis, T. (1994). The TPTP Problem Library. Proc. CADE-12, Nancy, LNAI 814, pp. 252--266.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC