De Raedt, L. and W. Van Laer: 1995, `Inductive Constraint Logic'. In: K. P. Jantke, T. Shinohara, and T. Zeugmann (eds.): Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, Vol. 997 of Lecture Notes in Artificial Intelligence. pp. 80--94, Springer-Verlag.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....ILP systems, the various types of imperfections that can appear in the data, the structure of the hypothesis space and, finally, the techniques that can be adopted for learning. Some of the most representative ILP systems are then described: GOLEM [MF90] FOIL [Qui90a] mFOIL [Dze91] and ICL [DRL95] Chapter 4 considers the problem of learning from incomplete information in the background and describes the adoption of abductive logic programs as the representation formalism. First, abductive logic programs are defined, together with a semantics and a proof procedure for them. A learning ....

....gorilla(rudolph)g When some negative interpretations are also given, the aim of the system is to find a theory that discriminate positive from negative interpretations, thus expressing regularities on positive interpretations that are false for negative ones. An example of such a system is ICL [DRL95] The following is an example of learning from interpretations from positive and negative examples. Example 26 Suppose we have the same positive observations and background knowledge as example 25, plus the following two sets of negative observations 1 = ffemale(liz) male(liz)g 2 = ....

[Article contains additional citation context not shown here]

L. De Raedt and W. Van Lear. Inductive constraint logic. In Proceedings of the 5th International Workshop on Algorithmic Learning Theory, 1995.

....users familiar with the propositional counterpart. 1.5. ORGANIZATION OF CHAPTERS 9 In Chapter 4 we apply the upgrading method in practice, starting from the well known CN2 rule learning algorithm by [Clark and Niblett, 1989; Clark and Boswell, 1991] and ending in the first order rule learner ICL [De Raedt and Van Laer, 1995]. We show how first order examples and hypotheses can be expressed in practice, and discuss the declarative language bias )LAB used in ICL. From an implementation point of view, we take a closer look at the design and structure of ICL and look at some practical issues. Finally, we briefly discuss ....

....and CNF expressions is discussed. Example 2.5 In Example 2.3 (page 17) the DNF hypothesis consists of 4 literals, whereas the CNF description contains only 3 literals 4. Note the rep lication of the proposition right black in the DNF hypothesis. This does not occur in the CNF description. O In [De Raedt et al. 1995], we have shown that for each DNF (CNF) de scription for a class, there exists a corresponding CNF (DNF) description of the complementary class, which has the same structure and complexity, but with V and positive negative literals switched. Indeed, given a complete and consistent DNF (CNF) ....

[Article contains additional citation context not shown here]

L. De Raedt and W. Van Laer. Inductive con- straint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Artificial Intelligence, pages 80-94. Springer-Verlag, 1995.

....a clause space for hypotheses that respect a particular syntax, the hypothesis language, specified by a learning bias. The selected theories, i.e. sets of clauses, must further be complete (they must cover all the positive examples) and correct (they must cover no positive example) ICL [RL95] proposes a high level concept specification language called DLAB in which the hypothesis language syntax can be defined. DLAB grammars are preprocessed in order to generate candidate hypotheses from the most general to the specific ones. We have used several DLAB specifications in order to ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Proceedings of the 5th Workshop on Algorithmic Learning Theory, Lecture Notes in Artificial Intelligence, 1995.

....makes it relatively easy to learn knowledge adapted to the problem at hand. Precisely, we show how to play with bias specifications in order to learn concept definitions enjoying such di#erent properties as robustness, readability or recognition e#ciency. DLAB, the declarative bias language of ICL [12], has reveal quite useful and flexible to achieve this goal. The first section gives some basic knowledge about cardiac arrhythmias. The next section presents the data and learning materials. Next, we describe the results obtained on learning five arrhythmias. Finally, we conclude and give some ....

....covers all the given positive examples) and is consistent (it covers no negative examples) LH is the hypothesis language and is generally a subset of first order logic. An interesting feature of ILP systems is to provide the users with declarative tools which provide means to specify LH . ICL [12] proposes a high level concept specification language called DLAB in which the hypothesis language syntax can be defined. DLAB grammars are preprocessed in order to generate candidate hypotheses from the most general to the specific ones (under # subsumption) ICL enables also multi class ....

[Article contains additional citation context not shown here]

L. De Raedt and W. Van Laer. Inductive constraint logic. In Proceedings of the 5th Workshop on Algorithmic Learning Theory, LNAI, 1995.

....of Tilde except for the differences between parallel and serial execution as described in this text. The different procedures are compared pairwise for the following data sets: ffl SB (Simple Bongard) and CB (Complex Bongard) several artificially generated sets of so called Bongard problems (De Raedt Van Laer, 1995) (pictures are classified according to simple geometric patterns) SB contains 1453 examples with a simple underlying theory, CB 1521 examples with a more complex theory. ffl Muta: the Mutagenesis data set (Srinivasan et al. 1996) an ILP benchmark (230 examples) ffl ASM: a subset of 999 ....

De Raedt, L., & Van Laer, W. (1995). Inductive constraint logic. Proceedings of the Sixth International Workshop on Algorithmic Learning Theory (pp. 80-- 94). Springer-Verlag.

....12) The rest of algorithm is similar to the version without packs (See Fig. 2) 3. 3 Rule induction Although we focused on decision tree induction, almost everything said so far also applies to top down rule induction (e.g. FOIL [11] Progol [9] and to top down constraint induction (e.g. ICL[7]) A top down rule induction system tries to cover all positive examples by learning a disjunction of conjunctive rules. Each time a new rule is learned, the systems removes the covered positive examples from the data set and tries to learn a next rule until all positive examples are covered or no ....

L. De Raedt and W. Van Laer. Inductive constraint logic. Unpublished, 1995.

....tree. A rst order decision tree [1] is a binary decision tree with conjunctions of rst order literals in the nodes. The leaves contain class values in case of a classi cation task or (vectors of) real values in case of a regression task. An example tree grown on one of the Bongard data sets [6] is shown in Fig. 1. The prediction task for this set is classifying pictures containing circles, squares and triangles as positive or negative. We use the learning from interpretations setting [5] in which each example is given by a set of (Prolog) facts. Notice that it is not necessary to ....

....times of a 10 fold run for serial (no optimisations) serial query packs, parallel, parallel intersection (share computations among di erent groups of trees) and nally parallel cross validation query packs. The data sets used are: The simple (SB) and complex (CB) Bongard data set [6] (this set was also used as running example in this text) SB contains 1453 examples with a 1 ACE is available for academic purposes upon request. http: www.cs.kuleuven.ac.be dtai ACE simple underlying theory, CB contains 1521 examples with a more complex theory. A subset (ASM) of 999 ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Articial Intelligence, pages 80-94. Springer-Verlag, 1995.

....Submitted to ECAI 98 March 6, 1998 those covered without assumptions. Note that we have considered a learning problem where no constraints are learned. In order to learn a full abductive theory, we learn first the rules, using the above algorithm, and then the constraints, using the system ICL [1] that learns from interpretations. The input interpretations for ICL are obtained from the set of assumptions generated in the rule learning phase. 3 Tasks The system can be successfully applied to solve a number of tasks that are difficult or impossible for most ILP systems: learning from ....

L. De Raedt and W. Van Lear, `Inductive constraint logic', in Proceedings of the 5th International Workshop on Algorithmic Learning Theory, (1995).

....on the basis of cumulativity alone, when looking at Table 1. While this result is far from conclusive, in our opinion it does provide motivation for further exploration of the issue. 5 An ILP algorithm biased towards cumulative features Most ILP algorithms (e.g. Progol [17] FOIL [19] ICL [4], are essentially rule set induction systems, and hence are biased towards non cumulativity. Some exceptions are tree based [1, 15] instance based [9] and probabilistic systems (e.g. Bayesian learners [10] Maccent [5] or Cussens probabilistic approaches [3] None of these seem to be ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Artificial Intelligence, pages 80--94. Springer-Verlag, 1995.

....aspects of these results. Introduction Early work in Inductive Logic Programming (ILP) included several systems that allowed the learner to ask questions in the learning process [24, 23, 18, 8] Most of recent work, however, tends to use learning from examples as the main paradigm (e.g. [20, 17, 9, 5]) Since only limited classes of expressions are learnable from examples [6] heuristics are used to obtain good results in practice. One recent strand of theoretical work has shown that larger classes of expressions are learnable if the learner is allowed to ask questions [12, 21, 4, 22, 16, 14, ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Proceedings of the International Conference on Algorithmic Learning Theory, 1995.

....(numerical values, nite domain values, intervals, To easily introduce negative examples like ( x 6= 0] p(x) specifying with a single constrained atom an in nite set of counter examples for property p. Consequently, there is a lot of works addressing this problem : one may cite [7, 11, 2, 15, 10]. These works focus on the integration in classical inductive process of new formulas (the constraints) which will be treated with speci c algorithms. In this paper, we are mainly concerned by the formal setting of ILP within a constraint framework, whatever the underlying machinery would be. Of ....

L. de Raedt and V. Van Laer. Inductive constraint logic. In Proc. 6th Intern. Workshop on Algorithmic Learning Theory, pages 80-94. Springer Verlag, 1995.

....caused by the language in which the information is represented. In this paper we propose a method to deal with these problems by allowing attributes to be dependent on each other and by allowing the user to specify his prior knowledge. Recent machine learning algorithms (such as Tilde [2] Icl [8], Warmr [9] use a more expressive rst order (or multiple relation) representation. Some systems that do conceptual clustering on examples represented in rst order logic exist, but often still need some information in propositional form. e.g. the system TIC [3] builds clustering trees but uses a ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Articial Intelligence, pages 80-94. Springer-Verlag, 1995.

No context found.

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Arti cial Intelligence, pages 80-94. Springer-Verlag, 1995. 22

....data. This makes the ILP system easy to use and understand by users familiar with the propositional counterpart. This methodology is perhaps the most important lesson learned during the development of several inductive logic programming systems and results (including [21] Tilde [9, 7] ICL [23], Claudien [20] Warmr [26] of the machine learning group in Leuven. The methodology starts from an existing propositional learner and provides a recipe for upgrading it towards the use of rst order logic. The recipe involves the use of examples which correspond to sets of ground facts ....

....from the original system. Following the methodology, it should be possible to turn virtually any propositional symbolic learner into an inductive logic programming system. To show how the methodology works, we demonstrate it on upgrading the well known CN2 [14, 13] learning algorithm towards ICL [23]. In Section 6 we give an overview of other systems that follow the same methodology. The paper is structured as follows: we rst elaborate on the characteristics of the propositional and the rst order knowledge representation and we show how the relational representation can overcome ....

[Article contains additional citation context not shown here]

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Arti cial Intelligence, pages 80-94. Springer-Verlag, 1995.

....for distances between objects in an attribute value representation (see e.g. 9, ch. 4] Recently there is a growing interest in using more expressive first order representations of objects and in upgrading propositional learning systems into first order learning systems (e.g. TILDE [2] ICL [5] and CLAUDIEN [4] Some ad hoc similarity measures exist for distances between first order objects [6] but they do not have all the desirable mathematical properties (e.g. the triangle inequality and the positive definiteness property) as a consequence their use may lead to ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Proceedings of the 5th Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Artificial Intelligence. Springer-Verlag, 1995.

....Employ transformation and record results Fig. 9. Experimental method for uncontrolled experiments Note on datasets We used two datasets for these experiments, which have the following properties: Predictive Toxicology Evaluation (PTE) 32] 337 examples; nondeterminacy estimate: 10 Bongard [9]: 5058 examples; nondeterminacy estimate: 2 Nondeterminacy estimates are based on the typical use of predicates that occur in the data; for instance, the molecules used in the PTE dataset contain on average some 30 atoms; introducing a new atom literal with a free variable identifying the atom ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Articial Intelligence, pages 80-94. Springer-Verlag, 1995.

....most specific hypothesis that covers (explains) all the examples. e.g. in Figure 1(b) the hypothesis H characterizes all the given examples. The most popular descriptive data mining technique is that of discovering association rules [2, 1, 24] The three companion systems, Claudien [13, 12] ICL [14] and Tilde [5, 6] can be considered first order upgrades of existing attribute value data mining approaches. Claudien upgrades the descriptive association rule approach, ICL upgrades the predictive production rule approach as incorporated in, e.g. CN2 [10, 9] and AQ [19] and Tilde is a recent ....

....It should be mentioned that the above description of ICL and CN2 is largely simplified. First, it may be the case that the data are noisy in which case a perfect solution may not exist. Under such conditions the algorithms will employ heuristics to find an approximation of the target concept (cf. [14]) Second, the find rule algorithm as outlined above performs hill climbing : it keeps track of a single current best candidate. ICL and CN2, however, perform a beam search, where one always keeps track of the m best rules. Finally, the above algorithm does not specify which literals can be added ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, pages 80--94. Springer-Verlag, 1995.

....we develop a methodology for upgrading propositional learners towards first order logic and demonstrate it at work. This methodology is perhaps the most important lesson learned during the development of several inductive logic programming systems and results (including [21] Tilde [9, 7] ICL [23], Claudien [20] Warmr [26] of the machine learning group in Leuven. The methodology starts from an existing propositional learner and provides a recipe for upgrading it towards the use of first order logic. The recipe involves the use of examples which correspond to sets of ground facts ....

....from the original system. Following the methodology, it should be easy to turn virtually any propositional symbolic learner into an inductive logic programming system. To show how the methodology works, we demonstrate it on upgrading the well known CN2 [14, 13] learning algorithm towards ICL [23]. In Section 6 we give an overview of other systems that follow the same methodology. The paper is structured as follows: we first elaborate on the characteristics of the propositional and the first order knowledge representation and we show how the relational representation can overcome ....

[Article contains additional citation context not shown here]

L. De Raedt and W. Van Laer. Inductive constraint logic. In Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, pages 80--94. Springer-Verlag, 1995.

....1993a) and the rule induction program RIPPER (Cohen 1995) For regression, the regression tree induction program M5 (Wang and Witten 1997) a reimplementation of M5 (Quinlan 1993b) was used. It can construct linear models in the leaves of the tree. Relational learning systems applied include ICL (De Raedt and Van Laer 1995), which induces classification rules, SRT (Kramer 1996) and TILDE (Blockeel and De Raedt 1998) The latter are capable of inducing both classification and regression trees. ICL is an upgrade of CN2 (Clark and Boswell 1991) to first order logic, TILDE is an upgrade of C4.5, and SRT is an upgrade of ....

De Raedt, L. and Van Laer, W. 1995. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the 6th International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Artificial Intelligence, pages 80--94. Springer-Verlag.

....learners towards first order logic, using the propositional learning strate1 Figure 1: Bongard Problem 47 gies. In this way, we can profit from the existing research on propositional learners and inherit its efficiency. As a case study we do a rational reconstruction of the ILP learner ICL [10, 20]. ICL, which stands for Inductive Classification Logic, is a first order upgrade of the propositional learner CN2 [6, 5] The original development of ICL did not exactly obey the 5 step methodology we present in this paper. However, analyzing ICL now reveals important lessons that also apply to ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the 6th International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Artificial Intelligence, pages 80--94. Springer-Verlag, 1995.

....be extended. 4 Optimisation of individual queries 4.1 Generalisations The cut transformation divides a body statically in independent groups of literals and each group itself contains literals that all depend on each other. A small 2 A similar optimisation was also implemented in the ICL system [9]. It can be controlled through the setting simplify. example will suffice to show that even in such a group, one can make further optimisations: we make the assumption that head variables are ground on the call, so that for further explanation they are ignored (and in fact have been removed) ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Artificial Intelligence, pages 80--94. Springer-Verlag, 1995.

....We stress, however, that the methodology of upgrading KDD techniques is not specific for Tilde, nor for induction of decision trees. It can also be used for rule induction, discovery of association rules, and other kinds of discovery. Systems such as Claudien (De Raedt and Dehaspe, 1997) ICL (De Raedt and Van Laer, 1995) and Warmr (Dehaspe and De Raedt, 1997) are illustrations of this. Both learn from interpretations and upgrade propositional techniques. ICL learns first order rule sets, upgrading the techniques used in CN2, and Warmr learns a first order equivalent of association rules ( association rules over ....

....of data mining systems to data sets; this includes ILP systems. KEPLER was deliberately designed to be very open, and systems using the learning from interpretations setting can be plugged into it as easily as other systems. At this moment few systems use the learning from interpretations setting (De Raedt and Van Laer, 1995; De Raedt and Dehaspe, 1997; Dehaspe and De Raedt, 1997) Of these the research described in (Dehaspe and De Raedt, 1997) the Warmr system: finding association rules over multiple relations; see also Dehaspe and Toivonen s contribution in this issue) is most closely related to the work described ....

L. De Raedt and W. Van Laer. Inductive constraint logic. In Klaus P. Jantke, Takeshi Shinohara, and Thomas Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Artificial Intelligence, pages 80--94. Springer-Verlag, 1995.

No context found.

De Raedt, L. and W. Van Laer: 1995, `Inductive Constraint Logic'. In: K. P. Jantke, T. Shinohara, and T. Zeugmann (eds.): Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, Vol. 997 of Lecture Notes in Artificial Intelligence. pp. 80--94, Springer-Verlag.

No context found.

L. De Raedt and W. Van Laer. Inductive constraint logic. In K. P. Jantke, T. Shinohara, and T. Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Arti cial Intelligence, pages 80-94. Springer-Verlag, 1995.

No context found.

L. De Raedt and W. Van Laer. Inductive constraint logic. In K. P. Jantke, T. Shinohara, and T. Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Arti cial Intelligence, pages 80-94. Springer-Verlag, 1995.

No context found.

De Raedt, L., & Van Laer, W. (1995). Inductive constraint logic. Proceedings of the 5th Workshop on Algorithmic Learning Theory (ALT-95). Springer-Verlag.

No context found.

L. De Raedt and W. Van Laer. Inductive constraint logic. In K. P. Jantke, T. Shinohara, and T. Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Arti cial Intelligence, pages 80-94. Springer-Verlag, 1995.

No context found.

L. De Raedt and W. Van Laer. Inductive constraint logic. In K. P. Jantke, T. Shinohara, and T. Zeugmann, editors, Proceedings of the Sixth International Workshop on Algorithmic Learning Theory, volume 997 of Lecture Notes in Arti cial Intelligence, pages 80-94. Springer-Verlag, 1995.

No context found.

) L. De Raedt and W. Van Lear. Inductive constraint logic. In Proceedings of the 5th International Workshop on Algorithmic Learning Theory, 1995.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC