J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....and separate and conquer rule learning systems in particular. One reason is that the rule sets have a natural and familiar first order version, namely Prolog predicates; the separate andconquer approach has been quite successful in learning various restricted sublanguages of Prolog [ Cohen, 1992a; Quinlan, 1990; Pazzani and Kibler, 1992 ] Rule sets are also representationally more powerful than competing representations such as decision trees; this is reflected in the fact that rule induction systems significantly outperform standard decision tree induction systems on many problems [ Pagallo and ....

....improve the asymptotic time complexity of rule induction with no apparent loss in accuracy. 2 Rule induction: does it scale up 2. 1 Noise free rule induction In this paper, we will use as our experimental testbed pFOIL, which is a propositional version of the rule induction system FOIL [ Quinlan, 1990 ] Briefly, pFOIL is a top down, separate and conquer rule induction system; it builds a rule set greedily by adding one rule after another to an initially empty rule set until all positive examples are covered. Each rule in turn is built using another greedy strategy, starting with a ....

[Article contains additional citation context not shown here]

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5(3), 1990.

....be used to represent the fact that there is a hyperlink on page1 that points to page2. In order to be able to deal with such a representation, one has to go beyond traditional attribute value learning algorithms and resort to inductive logic programming. Craven et al. 22] use a variant of Foil [54] to learn classification rules that can incorporate features from neighboring pages. The algorithm uses a deterministic version of relational path finding [57] which overcomes Foil s restriction to determinate literals [55] to construct chains of link to 2 predicates that allow the learner to ....

.... error pruning algorithm [36, 31] Before learning a rule, Ripper splits the available training data into a growing set (usually of the data) and pruning set (the rest of the data) It then learns single rules by greedily adding one condition at a time (using Foil s information gain heuristic [54]) until the rule no longer makes incorrect predictions on the growing set. Thereafter, the learned rule is simplified by deleting conditions as long as the performance of the rule does not decrease on the pruning set. All examples covered by the resulting rule are then removed from the training ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....could be used by the ILP system. To begin with, we defined two relations, with plans of implementing full Allen s interval logic [All83] The two relations were during(EventId1, EventId2) and after(EventID1, EventID2) Unfortunately, this approach did not work well for a number of reasons. FOIL [Qui90] GOLEM [MF92] and PROGOL [Mug95] were all tested on simplified datasets with unsuccessful results. Each did not succeed for di#erent reasons. The first was the size of the dataset. Even with a small number of events and training instances, the number of generated predicates was huge ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--267, 1990.

....of the inductive systems require many examples for learning complex definitions, since they do not use prior knowledge that can guide the search of generalized hypothesis. Some new techniques have been developed that use prior knowledge, but they are still mainly used for learning domain theories (Quinlan, 1990, Muggleton, 1992) instead of learning control knowledge. Similar work in lazy learning have been Lazy Explanation Based Learning, LEBL (Tadepalli, 1989) and Lazy Partial Evaluation, LPE (Clark and Holte, 1992) While LEBL refines the knowledge introducing exceptions, HAMLET modifies the ....

....explain correctly the problem solving choices from a single episode or from a static analysis of the domain definition. These speedup learning systems have been usually applied to problem solvers with the linearity assumption, such as the ones applied to Prolog or logic programming problem solvers (Quinlan, 1990, Zelle and Mooney, 1993, Muggleton, 1992) special purpose problem solvers (Langley, 1983, Mitchell et al. 1983) or other general purpose linear problem solvers (Etzioni, 1993, Fikes et al. 1972, Leckie and Zukerman, 1991, Minton, 1988, Prez and Etzioni, 1992) These problem solvers are known ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239-266, 1990.

....Individual analyses While ours is the first work to evaluate the combination of static and dynamic analyses, the two component techniques are well known. 8.1. 1 Dynamic analyses Dynamic analysis has been used for a variety of programming tasks; for instance, inductive logic programming (ILP) Qui90, Coh94] produces a set of Horn clauses (first order if then rules) and can be run over program traces [BG93] though with limited success. Programming by example [CHK 93] is similar but requires close human guidance, and version spaces can compactly represent sets of hypotheses [LDW00] Value ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....learned control rules. We consider this organization essential and part of the overall learning process [4] 5 Related work Most speedup learning systems have been applied to problem solvers with the linearity assumption, such as the ones applied to Prolog or logic programming problem solvers [15, 21], special purpose problem solvers [12, 9, 18] or other general purpose linear problem solvers [5, 10, 11, 14] These problem solvers are known to be incomplete and and incapable of finding optimal solutions. If we remove the linearity assumption, we are dealing with nonlinear problem solvers. ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....been experiencing in our more sophisticated recent tests. These situations are beneficial for our inductive learning style as they provide negative examples of the application of the learned rules. We are looking into applying and extending existing methods for relational based induction such as [ Quinlan, 1990 ] The hill climbing performance of our global learning algorithm will approach the ultimately correct control knowledge by converging gradually closer from both over specific and over general rule sets. Our learning algorithm reasons about and converges from points in the generalization space as ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....at the domain description [8] However, these strong deductive approaches invest a substantial explanation effort to produce correct control strategies from a single problem solving trace. Alternatively, inductive approaches acquire correct learned knowledge by observing a large set of examples [20, 26]. In this paper, we present hamlet, a system that learns control knowledge incremental and inductively. hamlet uses an initial deductive phase, where it generates a bounded explanation of the problem solving episode. Upon experiencing each new problem solving episode, hamlet refines its control ....

....the eager inductive and refinement modules will remove these features. To speed up the convergence of the learning, we are currently introducing more informed elaborated ways of removing and adding features from the description of the target concept, such as information gain measures, similarly to [20]. These procedure refine rule (rule) if covers negative examples p(rule) then if type(rule) deduced then refine deduced rule(rule) else for all rule1 in rules(target concept(rule) refine induced rule(rule1) else if deletedp(rule) then undelete rule(rule) procedure refine deduced rule ....

[Article contains additional citation context not shown here]

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....perMCC Non Confidential 10 formance is a good measure of feature quality. Information theoretic algorithms [Luc83, BH89, GH90] avoid the asymmetry of back propagation, but no other algorithm has directly addressed the goals of reducing redundancy or generating a simple completion function. FOIL [Qui90] is really not comparable. Quinlan used it to learn intensional definitions of the given predicates in the Family Relations problem. This is much better than the extensional definitions learned here, but he did not construct any new predicates. CIGOL [MB88] learns new concepts from relational ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5(3), 1990.

....are introduced as examples. Space limitations preclude a detailed description of the builders currently implemented in the system. 3.3 The master learning algorithm The master learning algorithm used in the WhizBang Site Wrapper is shown in Figure 4. The algorithm is based on the FOIL system [Quinlan, 1990; Quinlan and Cameron Jones, 1993] and learns a DNF expression, the primitive elements of which are predicates. As in FOIL, the outer loop of the learning algorithm (the learnPredicate function) is a setcovering algorithm, which repeatedly learns a single rule p (actually a conjunction of ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5(3):239--266, 1990.

....learning tasks, notably, search for conjunctions of constraints that identify a class or part of a class. For this search problem, search usually starts with a general classifier and each search operator corresponds to the conjunction of a constraint to the classifier under construction [5, 11]. Proceedings of the 19th Australasian Computer Science Conference, Melbourne, Australia, January 31 February 2 1996, pp. 1 10. Such search is unordered as the order in which constraints are added to a classifier is not significant. In recent years there has been an increase in interest in the ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, Volume 5, pages 239--266, 1990.

....learning algorithms [Hin89] While many of these systems learn descriptions based on bit vectors or simple hierarchies, our work deals with logical descriptions. Here too there is a history, dating back (at least) to Plotkin [Plo71] and Shapiro [Sha83] and including the more contemporary foil [Qui90] and the body of work on inductive logic programming (ILP) Mug92] However, while most of these projects begin with an empty theory and attempt to learn a target logic program by adding new clauses, theory revision processes work by modifying a given initial theory (which can involve both ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning Journal, 5(3):239--66, August 1990.

....but by the role they play in going from the current state to the goal. The use of learning algorithm on top of rich, logical representations, relates this work to the work in Inductive Logic Programming. For example, foil is a system that learns first order rules from data in a supervised manner [21]. Like Rivest s learning algorithm, foil selects the best rules one by one, eliminating the examples that are covered, and iterating this procedure on the examples that are left. The main difference is in the way the best rules are selected. In Rivest s algorithm, this is done by an exhaustive ....

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....in the modes suggested by domain experts. 1 As we will see, these di#erences cause problems for some standard approaches to ILP, and also suggest some interesting extensions. After presenting some background material, we begin by comparing the behavior of two o# the shelf ILP systems: FOIL [ Quinlan, 1990; Quinlan and Cameron Jones, 1993 ] and FLIPPER [ Cohen, 1995b ] We observe that FLIPPER is generally faster and achieves better results; also, surprisely, both systems produce less accurate theories when the hypothesis space is restricted according to expert intuitions about the problem. Based ....

....to contain coupling relationships, some of which are (by hypothesis) meaningful predictors of faults. Adopting these modes thus is one means of injecting prior knowledge into the learning process. 2. 2 ILP Systems Our first experiments were carried out with two o# the shelf ILP systems, FOIL6.4 [ Quinlan, 1990; Quinlan and Cameron Jones, 1993 ] and FLIPPER [ Cohen, 1995b ] Later experiments were carried out with modified versions of these systems. We will now briefly describe FOIL and FLIPPER. 2.2.1 Background on ILP The ILP systems used in this paper learn logic programs in certain constrained ....

[Article contains additional citation context not shown here]

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5(3), 1990.

.... methods proved to be most effective (see e.g. Mingers, 1989] or [Esposito et al. 1993] Pre pruning heuristically deciding when to stop growing clauses and concepts has been present in Inductive Logic Programming (ILP) in the form of stopping criteria for quite some time (see e.g. Foil [Quinlan, 1990], mFoil [Dzeroski and Bratko, 1992] and Fossil [Furnkranz, 1994] The basic idea behind most post pruning methods is to learn a concept description on one part of the training instances and to subsequently delete several parts of this theory in order to improve performance on the remaining set. ....

....description is generated that perfectly explains all training instances. This will be subsequently gener2 alized by cutting off branches of the decision tree (as in [Quinlan, 1987] or [Breiman et al. 1984] In ILP, Pre Pruning has been common in the form of stopping criteria as used in Foil [Quinlan, 1990], mFoil [Dzeroski and Bratko, 1992] and Fossil (see section 2) Post Pruning was introduced to ILP with an adaptation of Quinlan s Reduced Error Pruning [Brunk and Pazzani, 1991] First the training set is split into two subsets: a growing set and a pruning set . A concept description explaining ....

John Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....to derive all of the positive and none of the negative training examples using the relations provided in the background knowledge. Many ILP algorithms including all of the algorithms discussed in this paper address this problem with the so called separate and conquer strategy used in Foil [Quinlan, 1990]. The basic idea behind this approach is to learn one rule after the other until all of the positive examples are covered by at least one rule. The rules are constructed by evaluating each variabilization of each relation in the background knowledge with a greedy heuristic like information gain ....

....been common in the form of stopping criteria, i.e. heuristics that determine when to stop adding conditions to a rule, or when to stop adding rules to the concept description. The most commonly used criteria are ffl Encoding Length Restriction: This heuristic used in the classic ILP system Foil [Quinlan, 1990] is based on the Minimum Description Length principle [Rissanen, 1978] It prevents overfitting the noise by learning only as long as the costs of encoding a theory together with its exceptions will not exceed the costs of encoding the examples as they are. ffl Significance Testing was first used ....

John Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....We performed two experiments to compare Fossil s performance to the performance of Foil. In the first series we compared the behavior of the two systems with 10 training sets of 100 instances each at different noise levels, which has been the standard procedure for evaluating many ILP systems [Quinlan, 1990, Dzeroski and Lavrac, 1991, Dzeroski and Bratko, 1992b, Muggleton et al. 1989] In the second experiment we evaluated both programs at a constant noise level of 10 , but with an increasing number of training instances. According to the results of the previous experiments we set C = 0:3 and never ....

.... exactly this theory was learned, while in the other two the literal A = C had been added to the first clause, which still gives a 97.98 correct theory (see Appendix) What seems to be responsible for the drastic increase in the complexity of the learned clauses is that Foil s stopping criterion [Quinlan, 1990] is dependent on the size of the training set. In the KRK domain it performs very well on sample sizes of 100 training examples. The more this number increases, the more bits are allowed for the theory to explain the data. However, more examples do not necessarily originate from a more complex ....

John Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

....all of the genes of the particular type present in the database were used except genes which contained unknown bases in any of the positions used in the training examples. Table 2 lists the number of positive and negative examples in each of the datasets. 4 The learning algorithm FOIL The foil [Quinlan, 1990] machine learning algorithm learns from data encoded as relations and outputs concepts in the form of simple Datalog programs. Precisely, the output format is sets of function free Datalog with negation clauses. foil is given input relations defined extensionally as lists of ground tuples. 2 ....

John R. Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

No context found.

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

No context found.

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239-- 266, 1990.

No context found.

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

No context found.

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, Volume 5, pages 239--266, 1990.

No context found.

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

No context found.

J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239--266, 1990.

No context found.

Ross Quinlan, Learning Logical Definitions from Relations, Machine Learning 5, 3, 1990, 239-266.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC