Quinlan, J.R. (1996): Learning First-Order Definitions of Functions. Journal of Artificial Intelligence Research, to appear.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....When there are more than two classes, it is necessary to learn a theory for each class. The question is how to apply these theories to a new example [30] We have employed two approaches for dealing with multiclass problems. The first one is the use of ordered rules, also named decision lists [23, 19]. In this case, the first rule that covers the example assigns its label to it. The learning process consists of generating a rule for the class with most uncovered examples and iterate until all the examples are covered. Figure 1 shows an example of such a decision list. The second approach, ....

J.Ross Quinlan. Learning first-order definitions of functions. JAIR, pages 139-- 161, October 1996.

....5 10 fold cross validations) Results are taken from [D2eroski et al. 1999] We have left out the results of the regression systems. iments 7. The following systems have been used: the propositional learners C4.5 [Quinlan, 1993] and RIPPER [Cohen, 1995] and the relational learners FFoIL [Quinlan, 1996], SRT [Kramer, 1996] ICL and TILDE. A short overview of the results can be found in Table 6.7. Accuracy is classification accuracy and Accuracy( 1) is the predictive accuracy where only misclassification by more than one class counts as an error (e.g. slow as fast, moderate as resistant, and ....

J.R. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139-161, 1996.

....about atoms and bonds, much like in the mutagenesis domain. In [34] experiments on the relational data (denoted R1) and 2 propositional versions of the data (denoted P1 and P 2) has been performed with the propositional classification systems C4.5 and RIPPER [15] and the relational learners FFoil [59], SRT [45] ICL and Tilde. A short overview of the results can be found in Table 9. Accuracy is classification accuracy and Accuracy ( 1) is the accuracy where only misclassification by more than one class counts as an error (e.g. slow as fast, moderate as resistant, ICL has only been ....

J. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161, 1996.

....of inducing both classification and regression trees. ICL is an upgrade of CN2 (Clark and Boswell 1991) to first order logic, TILDE is an upgrade of C4.5, and SRT is an upgrade of CART (Breiman et al. 1984) TILDE cannot construct linear models in the leaves of its trees; SRT can. Finally, FFOIL (Quinlan 1996) was also applied to the classification version of the problem. It used a representation (denoted R2) based on the atom and bond relations, designed to avoid problems with indeterminate literals. New predicates are introduced, which stand for conjunctions of the form atom(M;X;Element1; ....

Quinlan, J.R. 1996. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161.

....representation allows the implementation of e#cient learning systems. However, it also makes it di#cult to apply machine learning to domains where data is rich in structure. In addition, the underlying structural information may actually prove essential in inducing insightful concepts. As Quinlan [Qui96] puts it: Data may concern objects or observations with arbitrarily complex structure that cannot be captured by the values of a predetermined set of attributes . For example, in a chemical domain, one may need to consider molecules as individuals, where molecules are complex arrangements of ....

J.R. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161, 1996.

....satisfiability criterion is also known as consistency with the negative examples. Due to the use of a more expressive first order formalism, ILP techniques are proven to be more effective in tackling problems that require learning relational knowledge than traditional propositional approaches (Quinlan, 1990). There are two major approaches in the design of ILP learning algorithms: top down and bottom up. Both approaches can be viewed more generally as a kind of set covering algorithm. However, they differ in the way a clause is constructed. In a top down approach, one builds a clause in a general to ....

....a top down approach, one builds a clause in a general to specific order where the search usually starts with the most general clause and successively specializes it with background predicates according to some search heuristic. A representative example of this approach would be the Foil algorithm (Quinlan, 1990; Cameron Jones Quinlan, 1994) In a bottom up approach, the search begins at the other end of the space where it starts with the most specific hypothesis, the set of examples, and constructs the clauses in a specific to general order by generalizing the more specific clauses. A representative ....

[Article contains additional citation context not shown here]

Quinlan, J. R. (1996b). Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5, 139--161.

....The tuples are simply values for a set of variables and the background predicates operate on these variables. The job of the learner is to find a definition of the target relation in terms of the background predicates (and or itself) that covers the tuples known to belong to R but none that do not [7]. This is a well studied problem in machine learning. For our purposes, we use a simplified version of the FOIL algorithm described in [7] although any learning algorithm that is capable of learning relations between attributes is applicable. In particular, we cast the problem as follows. The ....

....find a definition of the target relation in terms of the background predicates (and or itself) that covers the tuples known to belong to R but none that do not [7] This is a well studied problem in machine learning. For our purposes, we use a simplified version of the FOIL algorithm described in [7], although any learning algorithm that is capable of learning relations between attributes is applicable. In particular, we cast the problem as follows. The variables are the start and end times of operators and the background predicates are: ffl BEFORE(A,B) true if event A occurs before event ....

J. R. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161, 1996.

....10 or fewer) together with their optimal solutions. We will describe how positive and negative examples of the target concepts used by the control rules are heuristically extracted from the input data. The rules are generated by an inductive logic programming approach based on the FOIL algorithm (Quinlan, 1990; 1996); unlike much work in inductive logic programming, however, explicit background knowledge in the form of defined predicates is not supplied to the system. Experimental evaluation on five di#erent benchmarks domains from a recent planning competition is quite promising: training time is short (on ....

....and e#ects overlap with those of actions that do occur. 2. 3 Rule Induction Control rules are generated from the training examples by a greedy general to specific search in the space of restricted temporal logic programs, as shown in Table 2, using an algorithm based on the FOIL procedure (Quinlan, 1990; 1996). The simple temporal logic programs we consider are constructed as follows. There are three kinds of predicates: static predicates, which refer to facts whose truth cannot be changed by any action (e.g. the predicate in city used to assert that a particular location is part of a particular ....

[Article contains additional citation context not shown here]

Quinlan, J. R. (1996). Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5 , 139--161.

....and TILDE [1] The latter are capable of inducing both classification and regression trees. ICL is an upgrade of CN2 [5] to first order logic, TILDE is an upgrade of C4.5, and SRT is an upgrade of CART [3] TILDE cannot construct linear models in the leaves of its trees; SRT can. Finally, FFOIL [18] was also applied to the classification version of the problem. It used a representation (denoted R2) based on the atom and bond relations, designed to avoid problems with indeterminate literals. New predicates are introduced for conjunctions of the form atom(M;X;Element1; bond(M; X;Y; ....

Quinlan, J.R. 1996. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161.

....in this system are not in functional form. Each rule is an expression that defines an object. In this paper, we report on the results of an investigation into learning object classification rules by using an Inductive Logic Programming system which has been specialised to learn functions: FFOIL [22]. This means that target concepts in FFOIL can only be functions, that is their last argument is a function of other arguments. FOIL can also learn concepts in functional form, however FFOIL is specialised to handle these cases more efficiently. In this research, we investigate the possibility of ....

....object models. In that research, we have tried to learn a rule for each object, for example, mug(A) which tells us when an object A is a mug. The system exhibited good performance, and the results were quite interesting. In this paper, we have used one of the derivatives of FOIL, that is FFOIL [22] which has been specialised to learn First Order definitions of functions. In FFOIL, the target relation to be learned is of the form: F (A 1 ; A 2 ; An ) 1) in which the last argument An is a function of the previous arguments. In the current system, we have used images of five kinds of ....

J. R. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research (JAIR), 5:139--161, Oct. 1996.

.... then tested the system on two ILP benchmarks: the finite element mesh design problem introduced by Dolsak and Muggleton (1992) and a selection of the list processing programs from Bratko (1990) previously used by Quinlan and Cameron Jones (1993) We compare Foidl s performance to Foil and to FFoil (Quinlan, 1996), a recent specialized version of Foil for learning single output functions without explicit negative examples. First order decision lists enable Foidl to achieve accuracy on the finite element mesh design problem that is generally superior to a range of previous ILP systems. Foidl s ....

....variables introduced in the body. The gain metric evaluates literals based on the number of positive and negative tuples covered, preferring literals that cover many positives and few negatives. The papers referenced above provide details and information on additional features. 2. 3 FFOIL FFoil (Quinlan, 1996) is a descendant of Foil with modifications, somewhat similar to Foidl s, that specialize it for learning functional relations. 3 First, FFoil assumes that the final argument of the relation is an output argument and that the other arguments of 1 For further information on these systems and ....

[Article contains additional citation context not shown here]

Quinlan, J. R. (1996). Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5, 139--161.

....and assures objective evaluation of conditional probabilities. Keywords:discovery, data mining, error measure, conditional probability Rule Discovery: Error Measures and Conditional Rule Probabilities 1 1 Introduction Many promising rule discovery algorithms have been proposed [1] 8] 14] [27], 28] 29] 37] Most of these algorithms use their proprietary methods to discover rules. They use their own methods to . measure the error (or goodness) of rules; build the search space; and . handle or estimate the accuracy of the result. On the other hand, these algorithms share some ....

....definitions of a collection of background relations. Examples of tuples known to be in the target relation are provided and, in most cases, so are examples of tuples known to be not in Rel. A typical relational learning algorithm, FOIL, is summarized as follows. The language in which FOIL [28] [27] expresses theories is a restricted form of Prolog. It is essentially the Datalog language specified by Ullman [32] except that there is no requirement that all variables in a negative literal appear also in the head or in a positive literal. FOIL interprets not using negation as failure [3] ....

J. R. Quinlan, "Learning First-Order Definitions of Functions", to appear in JAIR (ftp://ftp.cs.su.oz.au/pub/ml/q.ffoil.ps)

....separately (for example, Gilmore and Self, 1988; Desmoulins and Van Labeke, 1996) However, there has not been a comparative study of these two types of learning systems on a single subject domain. This paper presents a study in which a zeroth order learner, C4.5, and a first order learner, FFOIL (Quinlan, 1996), were used as induction engines in a student modelling system and elementary subtraction was used as the subject domain. There are a number of reasons for choosing C4.5 and FFOIL to conduct this study. Among the state of the art machine learning alternatives, both C4.5 and FOIL are readily ....

Quinlan, J. R. (1996). Learning first-order definitions of functions. Journal of Artificial Intelligence Research 5:139--161.

....of completeness within the example set. One form of completeness explored is for functional relations. That is, given the input arguments of the relation the outputs should be also presented in the example set. This idea is used in the systems FILP ( 1] INDICO ( 11] and more recently in FFOIL ([9]) to avoid the explicit specification of negative examples. A more general approach within this direction not restricted to functional relations is proposed in [13] The second direction within the area of learning from positive examples is the Bayesian one. The Progol4.2 system ( 6] can learn ....

Quinlan, J.R. Learning First-Order Definitions of Functions, Journal of Artificial Intelligence Research, 5 (1996), 139-161.

....tuples of arguments that do not satisfy the predicate [10, 15] Since GP does generally not utilize negative examples, fairly comparing it to such ILP methods is difficult. However, several ILP systems have recently been developed that induce functions from only positive examples of I O pairs [2, 9, 13]. Consequently, we selected several of these methods, Foidl, IFoil, and FFoil, to allow for a direct comparison to GP. Further comparison of these three ILP system is presented in [4] 2.1 FOIL Since all of these systems are variations on Foil [15] we first present a brief overview of this ....

....that any computed ouput which does not match the single, correct output for the function represents a covered negative example. IFoil (Intensional Foil) is just Foidl without the use of decision lists. The code for these systems is available at http: www.cs.utexas.edu users ml. 2. 3 FFOIL FFoil [13] is a descendant of Foil with modifications similar to Foidl s, that specialize it for learning functional relations. First, FFoil assumes that the final argument of the relation is an output argument and that the other arguments of the relation uniquely determine the output argument. This ....

J. R. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161, 1996.

....the Foidl algorithm is useful for more than the problem for which it was originally designed. We have tested the system on two standard ILP problems: the finite element mesh design introduced by Dolsak and Muggleton (1992) and a selection of the list processing programs from Bratko (1990) used by Quinlan and Cameron Jones (1993). We compare Foidl s performance to Foil and to FFoil (Quinlan, submitted) a version of Foil which learns single output functions. The first order decision lists enable Foidl to achiever better accuracy on the finite element mesh design problem than has previously been reported for an ILP ....

....and list processing problems. Section 5 discusses related work, and Section 6 summarizes and presents our conclusions. 2 Background 2. 1 FOIL Since Foidl is based on Foil, we present a brief review of this important ILP system; see articles on Foil for a more complete description (Quinlan, 1990; Quinlan Cameron Jones, 1993; Cameron Jones Quinlan, 1994) 1 Foil learns a function free, first order, Hornclause definition of a target predicate in terms of itself and other background predicates. The input consists of extensional definitions of these predicates as tuples of constants of specified types. Foil also ....

[Article contains additional citation context not shown here]

Quinlan, J. R. (submitted). Learning first-order definitions of functions. Artificial Intelligence. Quinlan, J. R., & Cameron-Jones, R. M. (1993). FOIL: A midterm report. In Proceedings of the European Conference on Machine Learning, pp. 3--20 Vienna.

....predicted. Because this number depends not only on the edge itself, but also on neighbouring edges, the learning task is a typical ILP task. Table 3 compares Tilde s performance on this dataset with several state ofthe art systems. FOIL [ Quinlan, 1993b ] is a general purpose ILP system. FFOIL [ Quinlan, 1996 ] is a variant of it that can only learn functional definitions, but is very good at that. FORS [ Karalic, 1995 ] is also specialized for learning functional definitions. Indigo [ Geibel and Wysotzki, 1996 ] uses the transformational approach to ILP: it first transforms the learning data into a ....

....Indigo [ Geibel and Wysotzki, 1996 ] uses the transformational approach to ILP: it first transforms the learning data into a propositional representation. Then, a propositional decision tree induction system is used for the actual induction process. Figures for this table were copied from [ Quinlan, 1996 ] FOIL, FFOIL, FORS) and [ Geibel and Wysotzki, 1996 ] Indigo) 6.3 Musk Dataset The musk dataset was studied by Dietterich et al. Dietterich et al. 1996 ] who donated it to the UCI repository [ Merz and Murphy, 1996 ] Dietterich used these data to study the so called multiple instance ....

J. R. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161, October 1996.

....invent new predicates, or to induce Prolog programs with 3 These formulae can be expressed as Prolog queries, but only by ignoring floundering negation (using n which does not check the groundness of its arguments) cuts. The latter is equivalent to the induction of decision lists, e.g. FFoil[Qui96] Foidl[MC95] In the learning from interpretations setting, decision lists and logical decision trees have the same representational power (both can trivially be transformed into the other format) From the point of view of predicate invention, our work is related to Bain et al. s non monotonic ....

....trees are more expressive than the flat logic programs typically induced by ILP systems, and that this expressive power is related to the use of cuts or the invention of new predicates. This in turn relates our work to some of the work on induction of decision lists and predicate invention [BM92, Qui96, MC95] showing that these algorithms, too, have an expressivity advantage over algorithms inducing flat logic programs. From a practical perspective, we have developed the Tilde system, of which Quinlan s C4.5[Qui93a] is a special case (due to the learning from interpretations setting) ....

J. R. Quinlan. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161, October 1996.

....of a class of learning tasks in order to improve their efficiency and or effectiveness on such tasks. Examples of these classes include learning guard clauses for logic programs, learning definitions of functional relations, and learning definitions consisting of determinate clauses. ffoil [19] belongs in this second group, being an adaptation of foil for functional relations (and so strongly related to the work of Bergadano and Gunetti [2] A target relation R(V 1 ; V 2 ; Vn ) is functional if, for any ground values of v 1 , v 2 , v n Gamma1 of V 1 , V 2 , Vn Gamma1 , ....

....[20] Table 3 shows that boosting delivers a modest improvement in prediction accuracy. 6. 4 Past tense of English verbs Learning how to transform a verb from present to past tense was first studied in the connectionist community, but relational representations of the task have also been explored [11, 17, 19]. These experiments imitate Ling s [10] in using ten training and test sets of verbs taken from a corpus of 1500 phonetic verbs. This formulation used two target relations, delete(A; B) and add(A; C) which jointly state that the past tense of verb A is found by removing the string B from its end ....

Quinlan, J.R.: Learning first-order definitions of functions. Journal of Artificial Intelligence Research (1996) to appear

No context found.

Quinlan, J.R. (1996): Learning First-Order Definitions of Functions. Journal of Artificial Intelligence Research, to appear.

No context found.

Quinlan, J. R. (1996c). Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139 -- 161.

No context found.

Quinlan, J R "Learning first-order definitions of functions" Journal of Artificial Inteligence Reserach Volume 5 (1996) 139-161

No context found.

J. Ross Quinlan. 1996. Learning first-order definitions of functions. Journal of Artificial Intelligence Research, 5:139--161.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC