Results 1  10
of
41
Operations for Learning with Graphical Models
 Journal of Artificial Intelligence Research
, 1994
"... This paper is a multidisciplinary review of empirical, statistical learning from a graphical model perspective. Wellknown examples of graphical models include Bayesian networks, directed graphs representing a Markov chain, and undirected networks representing a Markov field. These graphical models ..."
Abstract

Cited by 276 (13 self)
 Add to MetaCart
This paper is a multidisciplinary review of empirical, statistical learning from a graphical model perspective. Wellknown examples of graphical models include Bayesian networks, directed graphs representing a Markov chain, and undirected networks representing a Markov field. These graphical models are extended to model data analysis and empirical learning using the notation of plates. Graphical operations for simplifying and manipulating a problem are provided including decomposition, differentiation, and the manipulation of probability models from the exponential family. Two standard algorithm schemas for learning are reviewed in a graphical framework: Gibbs sampling and the expectation maximization algorithm. Using these operations and schemas, some popular algorithms can be synthesized from their graphical specification. This includes versions of linear regression, techniques for feedforward networks, and learning Gaussian and discrete Bayesian networks from data. The paper conclu...
KnowledgeBased Artificial Neural Networks
, 1994
"... Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset informat ..."
Abstract

Cited by 185 (13 self)
 Add to MetaCart
Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN(KnowledgeBased Artificial Neural Networks) is a hybrid learning system built on top of connectionist learning techniques. It maps problemspecific "domain theories", represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several t...
Automated Refinement of FirstOrder HornClause Domain Theories
 MACHINE LEARNING
, 1995
"... Knowledge acquisition is a difficult, errorprone, and timeconsuming task. The task of automatically improving an existing knowledge base using learning methods is addressed by the class of systems performing theory refinement. This paper presents a system, Forte (FirstOrder Revision of Theories f ..."
Abstract

Cited by 93 (8 self)
 Add to MetaCart
Knowledge acquisition is a difficult, errorprone, and timeconsuming task. The task of automatically improving an existing knowledge base using learning methods is addressed by the class of systems performing theory refinement. This paper presents a system, Forte (FirstOrder Revision of Theories from Examples), which refines firstorder Hornclause theories by integrating a variety of different revision techniques into a coherent whole. Forte uses these techniques within a hillclimbing framework, guided by a global heuristic. It identifies possible errors in the theory and calls on a library of operators to develop possible revisions. The best revision is implemented, and the process repeats until no further revisions are possible. Operators are drawn from a variety of sources, including propositional theory refinement, firstorder induction, and inverse resolution. Forte is demonstrated in several domains, including logic programming and qualitative modelling.
Induction of FirstOrder Decision Lists: Results on Learning the Past Tense of English Verbs
 Journal of Artificial Intelligence Research
, 1995
"... This paper presents a method for inducing logic programs from examples that learns a new class of concepts called firstorder decision lists, defined as ordered lists of clauses each ending in a cut. The method, called Foidl, is based on Foil (Quinlan, 1990) but employs intensional background knowle ..."
Abstract

Cited by 69 (14 self)
 Add to MetaCart
This paper presents a method for inducing logic programs from examples that learns a new class of concepts called firstorder decision lists, defined as ordered lists of clauses each ending in a cut. The method, called Foidl, is based on Foil (Quinlan, 1990) but employs intensional background knowledge and avoids the need for explicit negative examples. It is particularly useful for problems that involve rules with specific exceptions, such as learning the pasttense of English verbs, a task widely studied in the context of the symbolic/connectionist debate. Foidl is able to learn concise, accurate programs for this problem from significantly fewer examples than previous methods (both connectionist and symbolic). 1. Introduction Inductive logic programming (ILP) is a growing subtopic of machine learning that studies the induction of Prolog programs from examples in the presence of background knowledge (Muggleton, 1992; Lavrac & Dzeroski, 1994). Due to the expressiveness of firstorder...
Using Qualitative Models to Guide Inductive Learning
, 1993
"... This paper presents a method for using qualitative modds to guide inductive learning. ..."
Abstract

Cited by 54 (2 self)
 Add to MetaCart
(Show Context)
This paper presents a method for using qualitative modds to guide inductive learning.
Inductive Synthesis of Recursive Logic Programs
, 1997
"... The inductive synthesis of recursive logic programs from incomplete information, such as input/output examples, is a challenging subfield both of ILP (Inductive Logic Programming) and of the synthesis (in general) of logic programs from formal specifications. We first overview past and present achie ..."
Abstract

Cited by 44 (7 self)
 Add to MetaCart
(Show Context)
The inductive synthesis of recursive logic programs from incomplete information, such as input/output examples, is a challenging subfield both of ILP (Inductive Logic Programming) and of the synthesis (in general) of logic programs from formal specifications. We first overview past and present achievements, focusing on the techniques that were designed specifically for the inductive synthesis of recursive logic programs, but also discussing a few general ILP techniques that can also induce nonrecursive hypotheses. Then we analyse the prospects of these techniques in this task, investigating their applicability to software engineering as well as to knowledge acquisition and discovery.
Connectionist theory refinement: Genetically searching the space of network topologies
 Journal of Artificial Intelligence Research
, 1997
"... An algorithm that learns from a set of examples should ideally be able to exploit the available resources of (a) abundant computing power and (b) domainspecific knowledge to improve its ability to generalize. Connectionist theoryrefinement systems, which use background knowledge to select a neural ..."
Abstract

Cited by 40 (1 self)
 Add to MetaCart
(Show Context)
An algorithm that learns from a set of examples should ideally be able to exploit the available resources of (a) abundant computing power and (b) domainspecific knowledge to improve its ability to generalize. Connectionist theoryrefinement systems, which use background knowledge to select a neural network's topology and initial weights, have proven to be effective at exploiting domainspecific knowledge; however, most do not exploit available computing power. This weakness occurs because they lack the ability to refine the topology of the neural networks they produce, thereby limiting generalization, especially when given impoverished domain theories. We present the REGENT algorithm which uses (a) domainspecific knowledge to help create an initial population of knowledgebased neural networks and (b) genetic operators of crossover and mutation (specifically designed for knowledgebased networks) to continually search for better network topologies. Experiments on three realworld domains indicate that our new algorithm is able to significantly increase generalization compared to a standard connectionist theoryrefinement system, as well as our previous algorithm for growing knowledgebased networks.
Inductive Logic Programming: derivations, successes and shortcomings
 SIGART Bulletin
, 1993
"... Inductive Logic Programming (ILP) is a research area which investigates the construction of firstorder definite clause theories from examples and background knowledge. ILP systems have been applied successfully in a number of realworld domains. These include the learning of structureactivity rules ..."
Abstract

Cited by 35 (5 self)
 Add to MetaCart
(Show Context)
Inductive Logic Programming (ILP) is a research area which investigates the construction of firstorder definite clause theories from examples and background knowledge. ILP systems have been applied successfully in a number of realworld domains. These include the learning of structureactivity rules for drug design, finiteelement mesh design rules, rules for primarysecondary prediction of protein structure and fault diagnosis rules for satellites. There is a well established tradition of learninginthelimit results in ILP. Recently some results within Valiant's PAClearning framework have also been demonstrated for ILP systems. In this paper it is argued that algorithms can be directly derived from the formal specifications of ILP. This provides a common basis for Inverse Resolution, ExplanationBased Learning, Abduction and Relative Least General Generalisation. A new generalpurpose, efficient approach to predicate invention is demonstrated. ILP is underconstrained by its logical ...
Combining Connectionist and Symbolic Learning to Refine CertaintyFactor Rule Bases
 Connection Science
, 1993
"... This paper describes Rapture  a system for revising probabilistic knowledge bases that combines connectionist and symbolic learning methods. Rapture uses a modified version of backpropagation to refine the certainty factors of a probabilistic rule base and it uses ID3's informationgain heur ..."
Abstract

Cited by 29 (3 self)
 Add to MetaCart
(Show Context)
This paper describes Rapture  a system for revising probabilistic knowledge bases that combines connectionist and symbolic learning methods. Rapture uses a modified version of backpropagation to refine the certainty factors of a probabilistic rule base and it uses ID3's informationgain heuristic to add new rules. Results on refining three actual expert knowledge bases demonstrate that this combined approach generally performs better than previous methods. 1 Introduction In complex domains, learning needs to be biased with prior knowledge in order to produce satisfactory results from limited training data. Recently, both connectionist and symbolic methods have been developed for biasing learning with prior knowledge (Shavlik and Towell, 1989; Fu, 1989; Ourston and Mooney, 1990; Pazzani and Kibler, 1992; Cohen, 1992). Most of these methods revise an imperfect knowledge base (usually obtained from a domain expert) to fit a set of empirical data. Some of these methods have been succ...
Rerepresenting and Restructuring Domain Theories: A Constructive Induction Approach
 Journal of Artificial Intelligence Research
, 1995
"... Theory revision integrates inductive learning and background knowledge by combining training examples with a coarse domain theory to produce a more accurate theory. There are two challenges that theory revision and other theoryguided systems face. First, a representation language appropriate for th ..."
Abstract

Cited by 28 (0 self)
 Add to MetaCart
(Show Context)
Theory revision integrates inductive learning and background knowledge by combining training examples with a coarse domain theory to produce a more accurate theory. There are two challenges that theory revision and other theoryguided systems face. First, a representation language appropriate for the initial theory may be inappropriate for an improved theory. While the original representation may concisely express the initial theory, a more accurate theory forced to use that same representation may be bulky, cumbersome, and difficult to reach. Second, a theory structure suitable for a coarse domain theory may be insufficient for a finetuned theory. Systems that produce only small, local changes to a theory have limited value for accomplishing complex structural alterations that may be required. Consequently, advanced theoryguided learning systems require flexible representation and flexible structure. An analysis of various theory revision systems and theoryguided learning systems ...