L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70:375--392, 1994.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....(for example, expected mean square error) A minor concern pertains to the fact that there may be no concept of negative examples when dealing with numerical data. Learning from positive examples only has recently been addressed within the ILP framework in various ways (see for example [25, 38, 37]) However, this has not proved a difficulty for the problems addressed here, and the changes to the implementation described in the next section adequately address these issues. Examples of their form and use are available in Appendix A. 3 Two changes to the implementation 3.1 Lazy evaluation ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAClearnable. Artificial Intelligence, 70:375--392, 1994.

....Figure 1: An example describing methane examples is proposed. Section 5 gives an outline for an implementation using this measure. Finally, in section 6 some conclusions are given. 2 Preliminaries 2. 1 Inductive Logic Programming We use the learning by interpretations setting in ILP (see also [4], 3] In this setting each example is represented by an interpretation. To make things tractable in datamining context, we don not use the full meaning of the logical program corresponding to these interpretations, but are only interested in the results of the queries (patterns) from some given ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAClearnable. Articial Intelligence, 70:375-392, 1994.

....deal with varied problems, large amounts of data and large hypotheses, and that it achieves good classification accuracy. 1. Introduction Work in Inductive Logic Programming (ILP) has established a core set of methods and systems that proved useful in a variety of applications (Muggleton De Raedt, 1994). Theoretical results, however, identified strong limits to learnability when only examples are used. One recent strand of theoretical work has shown that larger classes of expressions are learnable if the learner is allowed to ask questions (Arimura, 1997; Reddy Tadepalli, 1998; Rao Sattar, ....

....study the performance of the system by varying the size of programs and example set as well as comparing it to other systems. In all these good performance is demonstrated by LogAn H . 2. Learning from Interpretations Learning from interpretations has seen growing interest in recent years (De Raedt Dzeroski, 1994; De Raedt Van Laer, 1995; Blockeel De Raedt, 1998) Unlike the standard ILP setting, where examples are atoms, examples in this framework are interpretations of the underlying first order language (i.e. first order structures) We introduce the setup informally through examples. Formal ....

De Raedt, L., & Dzeroski, S. (1994). First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70, 375--392.

.... as a side effect derives a plan for these goals (McCarthy, 1958) Similarly, in partial order planning declarative information is given, and search in plan space is performed to find a plan (Weld, 1994) However, the problems involved in these approaches are computationally hard (Cook, 1971; Bylander, 1994). Recently, the approach has been generalized to handle stochastic domains, but as this is a generalization of the planning problem similar computational difficulties arise. Since the planning problem is computationally hard we cannot hope to find a new efficient solution to the problem. The main ....

....of EBL is discussed by DeJong and Bennett (1995) 8.7 Inductive Logic Programming The rules used in our strategies incorporate first order conjunctive conditions. The learning problem is therefore technically similar to that of Inductive Logic Programming (ILP) Muggleton, 1994; Muggleton De Raedt, 1994). However, the models differ in details that are crucial. One source of difference is the structure of examples. An example in the standard form of ILP (Quinlan, 1990; Dzeroski, Muggleton, Russell, 1992; Muggleton, 1994; Mooney Califf, 1995) includes a single ground instance of a relation and ....

[Article contains additional citation context not shown here]

De Raedt, L., & Dzeroski, S. (1994). First order jk-clausal theories are PAC-learnable.

....theory has dealt with learning of Boolean expressions in propositional logic. Early treatments of relational expressions were given by Valiant (1985) and Haussler (1989) but only recently more attention was given to the subject in the framework of Inductive Logic Programming (see e.g. Muggleton De Raedt, 1994; Cohen, 1995a; Nienhuys Cheng De Wolf, 1997) It is clear that the relational learning problem is harder than the propositional one and indeed except for very restricted cases it is computationally hard (Cohen, 1995b) To tackle this issue in the propositional domain various queries and oracles ....

....models is the use of background knowledge in the process of learning. This idea has been formalised in Inductive Logic Programming (ILP) where the background knowledge is given to the learner as a logical expression in the same language as that of the target expression being learned (Muggleton De Raedt, 1994). The background knowledge may be extensional, that is a set of ground facts, or intentional where it may include arbitrary expressions in the language. Finally, the framework of Learning to Reason (Khardon Roth, 1997) has been suggested for the study of systems that learn their knowledge in ....

[Article contains additional citation context not shown here]

De Raedt, L., & Dzeroski, S. (1994). First order jk-clausal theories are PAClearnable.

....declarative bias language, and heuristically searched in an iterative deepening fashion. Claudien has successfully discovered integrity constraints in many test applications [47] Both Index and Claudien are based on the alternative non monotonic semantics of ILP (also a result of the ILP project [21, 35, 15, 18]) In this semantics, individual clauses can be learned independently, which has been used to parallelize Claudien [16] Similarly, the ICDT system [19] is searching the space of general first order clauses, albeit using a more restricted declarative bias language and a simpler search strategy. ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are pac-learnable. Artificial Intelligence, 70:375--392, 1994.

....C. We denote the resulting function free definite clause by flat(C) and the set of unit clauses by defs(C) Rouveirol [14] has investigated the several properties of flattening. Muggleton [9, 11] has dealt with flattening in order to characterize his inverting implication. De Raedt and Dzeroski [2] have analyzed their PAC learnability of jk clausal theories by transforming possibly infinite Herbrand models into approximately finite models according to flattening. Recently, Nienhuys Cheng and de Wolf [13] have studied the properties of flattening with sophisticated discussion. Rouveirol [14] ....

....term appearing in C and v be a variable not appearing in C. Then, Cj v t denotes the definite clause obtained from C by replacing all occurrences of t in C with v. There exist several variants (but equivalent) of the definition of flattening: 1. Do we introduce an equality theory [9, 14] or not [2, 13] 2. Do we transform a constant symbol to an atom with an unary predicate symbol [2, 14] or not [13] As the definition of flattening, we adopt the definition similar as De Raedt and Dzeroski [2] that does not introduce an equality theory and does not transform a constant symbol. Let C be a ....

[Article contains additional citation context not shown here]

De Raedt, L. and Dzeroski, S.: First-order jk-clausal theories are PAC-learnable, Artificial Intelligence 90, 375--392, 1994.

....que les exemples la confirment, ou r eciproquement que l hypoth ese d ecrit les exemples. Ainsi, on distingue deux formes d induction, suivant le sens que l on donne a rendre compte : l induction explicative et l induction confirmatoire, egalement appel ee induction descriptive [ Muggleton et DeRaedt, 1994; Flach, 1995 ] 2.1 Induction explicative Etant donn ee une th eorie du domaine Th, l induction explicative vise a d eterminer les hypoth eses H qui expliquent, de fa con d eductive, les rapports d observations E, c est a dire les hypoth eses H telles que dans tous les cas o u H et Th ....

....premier cas, Caroline et P atrick partagent exactement les memes propri et es. Dans le second cas, Caroline et al..exandre partagent exactement les memes propri et es. 3. 2 Les approches existantes La plupart des approches existantes de la g en eralisation confirmatoire [ Helft, 1989; Marquis, 1992; De Raedt et Dzeroski, 1994 ] s ins erent dans un cadre appel e s emantique non monotone globale [ Muggleton et DeRaedt, 1994 ] Dans ce cadre, les g en eralisations confirmatoires sont les formules les plus g en erales qui sont vraies dans le mod ele de Herbrand minimal des observations M (E) D efinition 5 Etant ....

[Article contains additional citation context not shown here]

Luc De Raedt et Saso Dzeroski. First-order jkclausal theories are PAC-learnable. Artificial Intelligence, 70:375--392, 1994.

....papers cited above discuss the relation of transformation based learning to decision tree induction. We come back to the link with decision lists in the next section. B Warmr s logical setting for frequent pattern discovery is based on the learning from interpretations paradigm introduced in (De Raedt and Dzeroski, 1994). 3 The B Warmr algorithm Basic algorithm. Our description of the relational transformation based learning algorithm B Warmr is based on the notion of a confusion matrix M. As shown in Figure 2, given n classes, M is an (n Theta n) matrix, where M[i; j] equals the number of examples belonging ....

De Raedt, L., and Dzeroski, S. 1994. First order jk-clausal theories are PAC-learnable. Artificial Intelligence 70:375--392.

.... , tr(Tid,beer) In general, all work in ILP faces the theoretical result that evaluation of a query is an NP complete problem. However, queries with up to k atoms, where each atom contains with at most j terms, can be evaluated in polynomial time with respect to a relational database, cf. [17]. We now look at the different settings for frequent pattern discovery from three angles. In the language bias paragraph we show how Warmr can be tuned to simulate the restricted setting. In the candidate generation and evaluation paragraphs we review the basic ideas and principles underlying the ....

....discovery in databases, we refer to [23] 6. 1 Logical paradigm: learning from interpretations The definition of frequent query discovery and the (relatively) efficient candidate evaluation in Warmr is rooted in the learning from interpretations paradigm, introduced by De Raedt and Dzeroski [17] and related to other inductive logic programming settings in [13] Indeed, the r K s, described in Section 3.2.4 as partitions of the database, can be formalized in first order logic as Herbrand interpretations. Every r K in which a query succeeds is then a Herbrand model of that query. The ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAClearnable. Artificial Intelligence, 70:375--392, 1994.

....which correspond to a small relational database (or Prolog knowledge base) In other words, an example consists of multiple relations and each example can have multiple tuples for these relations. This setting is known in the literature as learning from interpretations and was first introduced in [15]. This representation is a natural upgrade of the attribute value representation where each example consists of a single tuple in a relational database. Learning from interpretations contrasts with the classical inductive logic programming setting learning from entailment which is employed by ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAClearnable. Artificial Intelligence, 70:375--392, 1994.

....literal restriction) In our example r0(X,Y,Z) r1(Y,U) r2(Z,R) r3(U,R) X = x0 R medium. would be a legal group description, assuming the object relation is r0, X is a nominal or tree structured attribute, R is an ordered attribute, and the set of foreign links is F = fr0[2] r1[1] r0[3] r2[1] r1[2] r3[1] r2[2] r3[2]g. On the other hand, a) r0(X,Y,Z) r3(U,R) b) r0(X,Y,Z) r3(X,R) c) r1(Y,U) r2(Z,R) r3(U,R) would not be legal subgroup descriptions because they are not linked (a) do not respect the foreign links (b) or do not start with the designated object ....

....implemented directly in the algorithm. For databasebased implementation, sampling would need be implemented directly in the database system to be efficient. 5 Related Work The discovery task defined in this paper is related to a number of other learning tasks that have been proposed in the past [3, 4, 13, 8], mostly in the field of ILP. The primary differences are in the instantiations of the evaluation function d and the requirements on the solution. First, since we use distributional unusualness d as evaluation function, which does not monotonically decrease when specializing, we uses minimal ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are pac-learnable. Artificial Intelligence, 70:375--392, 1994.

....q is a satisfaction predicate indicating whether the sentence of the language describes an interesting property of the data. This point of view has either implicitly or explicitly been used in discovering integrity constraints from databases, in inductive logic programming, and in machine learning [3, 4, 10, 11, 12]; some theoretical results can be found in [15] A suggested logical formalism for this approach is given in [9] where we argue that the logic L should be a slice of a probabilistic first order logic. In such querying of rules, the user often wants to cross the boundary between data and rules ....

L. De Raedt and S. Dzeroski. First-order jk-clausal theories are PAClearnable. Artificial Intelligence, 70:375 -- 392, 1994.

....Introduction One active area of research in machine learning is learning concepts expressed in firstorder logic. Since most researchers have used some variant of Prolog to represent learned concepts, this subarea is sometimes called inductive logic programming (ILP) Muggleton, 1992; Muggleton De Raedt, 1994). Within ILP, researchers have considered two broad classes of learning problems. The first class of problems, which we will call here logic based relational learning problems, are first order variants of the sorts of classification problems typically considered within AI machine learning ....

.... by the target clause as examples: specifically, the instance e = f; D) is covered by P; DB iff P DB (f D) As the example above shows, there is also a close relationship between extended instances and literals with function symbols that have been removed by flattening (Rouveirol, 1994; De Raedt Dzeroski, 1994). We have elected to use Datalog programs and the model of extended instances in this paper for several reasons. Datalog is relatively easy to analyze. There is a close connection between Datalog and the restrictions imposed by certain practical learning systems, such FOIL (Quinlan, 1990; Quinlan ....

[Article contains additional citation context not shown here]

De Raedt, L., & Dzeroski, S. (1994). First-order jk-clausal theories are PAC-learnable.

....in L that are sufficiently true in the data and furthermore fulfill the user s other criteria for interestingness. This point of view has either implicitly or explicitly been used in discovering integrity constraints from databases, in inductive logic programming, and in machine learning [6, 7, 26, 30, 32]; some theoretical results can be found in [37] and a suggested logical formalism in [23] While the frequency of occurrence of a pattern or the truth of a sentence can defined rigorously, the interestingness of patterns or sentences seems much harder to specify and measure. A general algorithm ....

....The problems are very varying, from architectural issues to specific algorithmic questions. For brevity, the descriptions are quite succinct, and I also provide only a couple of references. Framework and general theory 1. Develop a general theory of data mining. Possible starting points are [6, 7, 23, 26, 30, 37]. One might call this the theory of inductive databases. 2. What is the relationship between the logical form of sentences to be discovered and the computational complexity of the discovery task (The issue of logical form vs. sample size is considered in [27] 3. Prove or disprove the CPM ....

L. De Raedt and S. Dzeroski. First-order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70:375 -- 392, 1994.

....could mean that is true or almost true in r, or that defines (in some way) a sufficiently large or otherwise interesting subgroup of r. The roots of this approach are in the use of diagrams of models in model theory (see, e.g. 7] The approach has been used in various forms for example in [2, 8, 9, 16, 17, 21]. One should note that in contrast with, e.g. 8] our emphasis is on very simple representation languages. Obviously, if L is infinite and q(r; is satisfied for infinitely many sentences, an explicit representation of) Th(L; r; q) cannot be computed feasibly. Therefore for the above ....

L. De Raedt and S. Dzeroski. First-order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70:375 -- 392, 1994.

....then expression, where the expressions are, e.g. of the form A 40, B = 1, etc. Such rules can be found using the above algorithm. Several choices of the specialization relation are possible, and the number of iterations in the outermost loop of the algorithm depends on that choice. 2 Example 7 Consider the discovery of all inclusion dependencies that hold in a given database instance [ 12; 16; 19 ] Given a database schema R, an inclusion dependency (IND) over R is an expression R[X] S[Y ] where R and S are relation schemas of R, and X and Y are equal length sequences of attributes ....

L. De Raedt and S. Dzeroski. First-order jk- clausal theories are PAC-learnable. Artificial Intelligence, 70:375 -- 392, 1994.

....clause is as hard as learning boolean DNF. Together with positive results from the companion paper, these negative results establish a boundary of efficient learnability for recursive function free clauses. 1. Introduction Inductive logic programming (ILP) Muggleton, 1992; Muggleton De Raedt, 1994) is an active area of machine learning research in which the hypotheses of a learning system are expressed in a logic programming language. While many different learning problems have been considered in ILP, including some of great practical interest (Muggleton, King, Sternberg, 1992; King, ....

.... ; list123 ) and a description containing these atoms: components(list12,1,list2) components(list2,2,nil) components(list123,1,list23) components(list23,2,list3) components(list3,3,nil) The use of extended instances and function free programs is closely related to flattening (Rouveirol, 1994; De Raedt Dzeroski, 1994); some experimental learning systems also impose a similar restriction (Quinlan, 1990; Pazzani Kibler, 1992) Another motivation for using extended instances is technical. Under the (sometimes quite severe) syntactic restrictions considered in this paper, there are often only a polynomial number ....

[Article contains additional citation context not shown here]

De Raedt, L., & Dzeroski, S. (1994). First-order jk-clausal theories are PAC-learnable.

.... different notions of explanation have been employed in inductive learning, resulting in different frameworks for induction, most notably learning from interpretations (as studied in Valiant s pac learning framework, 24, 2] and in propositional or attribute value approaches to learning, but see [5]) and learning from entailment (as in inductive logic programming [17] and e.g. 10] In learning from interpretations, the aim is to induce a hypothesis (a logical formula) from a set E consisting of interpretations that are (resp. are not) a model for the unknown target theory. Learning from ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70:375--392, 1994.

....database with ExKey = i. Notice the ExampleKey is somewhat similar to notion of a clustering field as used in the database literature. The idea of partitioning the database also fits in the learning from interpretations paradigm (an interpretation here corresponds to a partition) introduced by [De Raedt and Dzeroski, 1994] and related to other inductive logic programming settings in [De Raedt, 1996b] From a practical point of view a significant speed up can be obtained if the partioning scheme creates examples that fit in main memory. As we will see later, the algorithm for efficiently mining association rules ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70:375--392, 1994.

.... general definition has emerged that can be found almost identically in most ILP papers addressing this task [Mug91, MDR94, LD94] On the other hand, existing work on defining the description learning task has concentrated exclusively on the so called non monotonic semantics of ILP [RB93] DRD94, D 95] strongly associating the task of finding descriptive properties of data with a closed world interpretation of the data in the spirit of Helft s formalization [Hel89] In this paper, we examine whether these restrictions can be relaxed and propose a general definition of the description ....

....(statements) is tested locally, i.e. on each dataset separately. In the family case, a single dataset comprises the facts about one family. The use of background knowledge that is common across all datasets is also allowed. Special cases of this setting are considered by De Raedt and Dzeroski [DRD94] and De Raedt and Van Laer [DRVL95] Definition 2 (Local nonmonotonic ILP) For a hypothesis language LH , the following problem is called the local nonmonotonic ILP problem. Given: ffl a set of positive examples E (datasets of interest) ffl a set of negative examples E Gamma ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are paclearnable. Artificial Intelligence, 70:375--392, 1994.

....where combining information from explanatory and descriptive ILP could be useful. We present some basic algorithmic frameworks for learning in the new framework, and report on some preliminary experiments with encouraging results. 1 Introduction Inductive logic programming (ILP, Muggleton and De Raedt, 1994 ] is concerned with learning clausal theories in first order logic. Two main approaches exist to learning in first order logic, known under the names of explanatory and descriptive learning. The first [ Muggleton, 1995 ] is also called learning from entailment or normal ILP, the second [ De ....

....1994 ] is concerned with learning clausal theories in first order logic. Two main approaches exist to learning in first order logic, known under the names of explanatory and descriptive learning. The first [ Muggleton, 1995 ] is also called learning from entailment or normal ILP, the second [ De Raedt and Dzeroski, 1994 ] is also called learning from interpretations or nonmonotonic ILP. The first setting is concerned with the induction of rules that explain (correctly classify) the given observations, whereas the latter is concerned with the induction of constraints that describe the (dependencies in the) given ....

[Article contains additional citation context not shown here]

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAClearnable. Artificial Intelligence, 70:375--392, 1994.

.... Ongoing Leuven Work The, from a practical point of view, most relevant recent results by the Leuven ILP group are concerned with the development of two efficient induction systems: CLAUDIEN [13] and ICL [15] This progress was made possible by theoretical advances in computational learning theory [14], in bias specification [1] and in the knowledge representation for examples and hypotheses used, i.e. by using the so called non monotonic semantics for inductive logic programming [60] The CLAUDIEN data mining system performs induction on observations belonging to a single class and discovers ....

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70:375--392, 1994.

....of this paper, it is sufficient to regard each example as a small relational database, i.e. as a set of facts. Within learning from interpretations, one may also specify background knowledge in the form of a Prolog program which can be used to derive additional features of the examples. 1 See [ De Raedt and Dzeroski, 1994; De Raedt, 1996; De Raedt et al. 1998 ] for more details on learning from interpretations. For instance, examples for the well known mutagenesis problem [ Srinivasan et al. 1996 ] can be described by interpretations. Here, an interpretation is simply an enumeration of all the facts we know ....

....leaf. If the leaves are coherent with respect to classes, this method would yield relatively high classification accuracy with a minimum of class information available. This is quite similar in spirit to Emde s method for learning from few classified examples, implemented in the COLA system [ Emde, 1994 ] A similar reasoning can be followed for regression, leading to unsupervised regression ; again this may be useful in the case of partially missing information. We conclude that clustering can extend classification and regression towards unsupervised learning. Another extension in the ....

[Article contains additional citation context not shown here]

L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAClearnable. Artificial Intelligence, 70:375--392, 1994.

No context found.

L. De Raedt and S. D#zeroski. First order jk-clausal theories are pac-learnable. Arti#cial Intelligence, 70:375#392, 1994.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC