Towell G., Shavlik J. 1993. Refining Symbolic Knowledge Using Neural Networks, Machine Learning: An Artificial Intelligence Approach, vol. IV.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....diagnostics problems. However, the rules performed by a fully connected network are hard to understand because there is huge number of its synaptic links. For training neural network, it is required also to collect the representative classified data set and spend an extensive computation time [9, 10, 16, 17]. The genetic based self organizing methods have been used to reduce both a neural network redundancy and learning time. In particular, the Group Method of Data Handling (GMDH) of Ivakhnenko was effectively exploiting to train a polynomial multi layered neural network of optimal complexity on ....

Towell G., Shavlik J. 1993. Refining symbolic knowledge using neural networks, Machine Learning: An Artificial Intelligence Approach, vol. 4, 71-101.

....used by ANNs. It is very difficult to interpret the knowledge encapsulated by the numerical weights of ANNs. Rule extraction from ANNs provides a mechanism to interpret this numerically encoded knowledge. Several rule extraction algorithms have been developed, including [Craven et al. 1993,Fu 1994,Towell 1994,Viktor 1998] These algorithms have shown to be efficient in the knowledge extraction process. The output of rule extraction algorithms are propositional DNF rules with attribute value tests of the form A i rel operator boundary value, e.g. petal width 49.50. Classification problems with ....

GG Towell and JW Shavlik, Refining Symbolic Knowledge using Neural Networks, Machine Learning, Vol. 12, 1994, pp 321-331.

....encode exactly those extracted rules. Once reset in this way, new instructions may be incorporated into the network and the process of inductive learning may begin again. Weight compilation approaches of this kind have been used to encode various forms of sentential rules [ McMillan et al. 1991; Towell and Shavlik, 1994 ] the transitions of finite state automata [ Giles and Omlin, 1993 ] and advice for an agent in an artificial environment [ Maclin and Shavlik, 1994 ] Unfortunately, none of these models provide a connectionist explanation for how instructions are compiled into the network. Also missing is ....

Geoffrey G. Towell and Jude W. Shavlik. Refining symbolic knowledge using neural networks. In Ryszard S. Michalski and Gheorghe Tecuci, editors, Machine Learning: A Multistrategy Approach, volume 4, pages 405--429. Morgan Kaufmann, San Mateo, 1994.

....therefore firmly in the logistic paradigm. The opposing paradigm of neural computation has its own wide following. In this approach the key notion is distributed and parallel activity in a network. Claims have been made for either paradigm that are presumably unachievable by the other. Recently [ 35 ] there have been attempts to exploit the relative strengths of each paradigm and combine them to improve performance. This technical report is a contribution toward this reconciliation. Here, we study a hybrid symbolic neural network architecture for belief representation called a Neural Logic ....

Geoffrey G. Towell and Jude W. Shavlik. Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, 1991.

....diagnostics problems. However, the rules performed by a fully connected network are hard to understand because there is huge number of its synaptic links. For training neural network, it is required also to collect the representative classified data set and spend an extensive computation time [9, 10, 16, 17]. The genetic based self organizing methods have been used to reduce both a neural network redundancy and learning time. In particular, the Group Method of Data Handling (GMDH) of Ivakhnenko was effectively exploiting to train a polynomial multi layered neural network of optimal complexity on ....

Towell G., Shavlik J. 1993. Refining symbolic knowledge using neural networks, Machine Learning: An Artificial Intelligence Approach, vol. 4, 71-101.

....combinations of antecedents from the initial rules, which makes this a comparatively difficult learning task for a system using Horn clauses. Finally, it should be noted that when Kbann translated its results into Horn clauses, the resulting theory was significantly more complicated than Either s [45]. This is because Either s goal is to produce a minimally revised Horn clause theory and Kbann has no such bias. 7.2 Soybean Results In order to demonstrate Either s ability to revise multiple category theories, Either was used to refine the expert rules given in [24] This is a theory for ....

G. Towell and J. Shavlik. Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, Harper's Ferry, W.Va., Nov. 1991.

....Machine Learning, Case Based Reasoning, Knowledge Refinement, Knowledge Based Neural Networks 1 Introduction Knowledge Based Neural Networks are concerned with the use of domain knowledge to determine the initial structure of Neural Networks. This knowledge can be supplied by the domain expert [7, 19] or induced by a machine learning program [9, 14] KBNN are shown to classify better unseen examples than randomly initialized NN. In this paper we study the potential of Case Based Reasoning for further improvement of a trained KBNN. The approach is inspired by successful use of CBR for ....

G.G. Towell and J.W. Shavlik. Refining Symbolic Knowledge Using Neural Networks, In: R. Michalski and G. Tecuci (eds.) Machine Learning, v.4, Morgan Kaufmann, pp. 405-430, 1994.

.... least M out of N properties of a certain kind are present in an object) Problems of this type occur in various real world problems, for example, in medicine (Spackman, 1988) planning (Callan Utgoff, 1991) game playing (Fawcett Utgoff, 1991) biology (Baffes Mooney, 1993) and biochemistry (Towell Shavlik, 1994). The proposed method addresses a class of problems that require learning descriptions combining one or more M of N concepts with one or more DNF expressions. Such combined descriptions are called conditional M of N rules. The well known M of N rules are thus a special case of conditional M of N ....

Towell, G. G. & Shavlik, J. W. (1994). Refining Symbolic Knowledge Using Neural Networks.

....it to Either and various other systems on refining the DNA promoter domain theory. 1 Introduction Recently, a number of machine learning systems have been developed that use examples to revise an approximate (incomplete and or incorrect) domain theory [ Ginsberg, 1990; Ourston and Mooney, 1990; Towell and Shavlik, 1991; Danyluk, 1991; Whitehall et al. 1991; Matwin and Plante, 1991 ] Most of these systems revise theories composed of strict if then rules (Horn clauses) However, many concepts are best represented using some form of partial matching or evidence summing, such as M of N concepts, which are true ....

....Neither (New Either) computes only the single best repair for example, and is therefore much more efficient. Also, because it was restricted to strict Horn clause theories, Either did not produce as accurate results as Kbann (a neural network revision system) on the DNA promoter problem [ Towell and Shavlik, 1991; Towell and Shavlik, 1992 ] Some aspects of the promoter concept fit the M of N format, since there are several potential sites where hydrogen bonds can form between the DNA and a protein; if enough of these bonds form, promoter activity can occur. Either attempts to learn this concept by ....

[Article contains additional citation context not shown here]

G. Towell and J. Shavlik. Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, pages 257--272, Harper's Ferry, W.Va., Nov. 1991.

.... of descriptions that involve counting properties (e.g. that M properties out of N possible properties are present in an object) which may be additionally combined with logical conditions (e.g. in the DNF form) Problems of this type occur in many real world problems (e.g. Spackman, 1988; Towell Shavlik, 1994). The proposed solution is based on the application of a new type of constructive induction rule, counting attribute generation rule, which explores an attribute symmetry in generated hypotheses. Such a symmetry is indicated by the presence of the exclusive or or equivalence patterns in the ....

Towell, G. G. and Shavlik, J. W., "Refining Symbolic Knowledge Using Neural Networks," in Machine Learning: A Multistrategy Approach , Vol. IV, Michalski, R.S. and G. Tecuci, Morgan Kaufmann, San Mateo, CA, pp. 1994.

....an existing correct program (e.g. Talus [Murray, 1988] are primarily useful in tutoring environments, since a correct program is rarely available in other situations. Most other work in theory revision is propositional in nature, and therefore inapplicable to logic programming [Ginsberg, 1990; Towell and Shavlik, 1991; Cain, 1991] Focl [Pazzani et al. 1991] uses an initial theory to guide a Foil based system; however, it produces a flat, operationalized definition instead of a revised theory. A version of Focl that performs theory revision has been developed [Pazzani and Brunk, 1990] however, it requires ....

G. Towell and J. Shavlik. Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, pages 257--272, Harper's Ferry, W.Va., Nov. 1991.

....in the form of a sequence of desired actions. Mahadevan and Connell initialize their reinforcement learner with a prespecified subsumption architecture, and Singh guides his reinforcement learner by giving it abstract actions to decompose. One of the most similar approaches to ours is that of Towell and Shavlik (1991). They also couple rule based input with a refinement method; however, their refinement method is neural networks. This multistrategy system converts rules into a network topology. The content of each rule is preserved; therefore, the transformation is syntactic. Our multistrategy system, on the ....

Towell, G. and J. Shavlik, Refining symbolic knowledge using neural networks. In Proc. of the Eighth International Workshop on Multistrategy Learning, 1991.

....efforts. A new generation of more effective machine learning algorithms have been developed which combine induction with other machine learning techniques. These methods use two inputs, examples plus an initial domain theory (Ginsberg, 1990; Ourston and Mooney, 1990; Craw and Sleeman, 1990; Towell and Shavlik, 1991). Such learners are termed theory refinement systems since they take an input specification (called the theory) and produce a revised version of that specification which is consistent with the examples. The idea is one of refinement; the learner starts with an initial theory that is incorrect or ....

Towell, G. and Shavlik, J. (1991). Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, pages 257--272.

....fit the M of N format. To demonstrate the advantages of Neither, we present experimental results from two real world domains. 1 Introduction Recently, a number of machine learning systems have been developed that use examples to revise an approximate (incomplete and or incorrect) domain theory [4, 11, 18, 3, 21, 7]. Most of these systems revise theories composed of strict if then rules (Horn clauses) However, many concepts are best represented using some form of partial matching or evidence summing, such as M of N concepts, which are true if at least M of a set of N specified features are present in an ....

....single best repair for each example, and is therefore much more efficient. Also, because it was restricted to strict Horn clause theories, the old Either algorithm could not produce as accurate results as a neural network revision system called Kbann on a domain known as the DNA promoter problem [18, 19]. Essentially, this is because some aspects of the promoter concept fit the M of N format. Specifically, there are several potential sites where hydrogen bonds can form between the DNA and a protein; if enough of these bonds form, promoter activity can occur. Either attempts to learn this concept ....

G. Towell and J. Shavlik. Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, pages 257--272, Harper's Ferry, W.Va., Nov. 1991.

.... contender would certainly be the growing focus on multistrategy learning [MAC 93; MIC 94] Though early efforts in this area combined purely symbolic [LEB 86; PAZ 88] or purely subsymbolic learning [HEC 87] there have been increasing attempts to integrate symbolic and subsymbolic approaches [TOW 94, KAI 94] Machine learning research at the University of Geneva reflects this global trend: since the late 1980s, the Artificial Intelligence Group of the Computer Science Department has been exploring the confluence of the symbolic and neural paradigms from three different perspectives. The ....

G.G. Towell & J.W. Shavlik. Refining Symbolic Knowledge Using Neural Networks. In [MIC 94].

....et al. 1986 ] and theory revision [ Mooney and Ourston, 1991 ] are one way to deal with this problem. In this approach, a score is calculated measuring how well an example matches each symbolic rule and the rule with the greatest score is invoked. Another approach is to use N of M rules [ Towell and Shavlik, 1991; Ginsberg et al. 1988 ] which fire if at least N of their M antecedents are satisfied, e.g. 3 of a; b; c; d C 1 . Unfortunately, this approach does not consider the relative strengths of each of the antecedents. Neural networks, on the other hand, use connection weights to encode relative ....

....lowweighted links are deleted. Backpropagation and node addition deletion continue in a cycle until all of the training examples are correctly classified. Once the network has been trained, the revised rules can be read directly off of the network no retranslation is necessary. Unlike KBANN [ Towell and Shavlik, 1991 ] the direct correspondence between weighted links and probabilistic rules removes any distinction between the symbolic and connectionist representations. The rest of this paper is organized as follows. Section 2 presents the Rapture algorithm in some detail. Section 3 shows some preliminary ....

[Article contains additional citation context not shown here]

G. Towell and J. Shavlik. Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, pages 257--272, Harper's Ferry, W.Va., Nov. 1991.

....these bad revisions can snowball and result in an overall decrease in performance. Introduction Recently, a number of machine learning systems have been developed that use examples to revise an approximate (incomplete and or incorrect) domain theory [ Ginsberg, 1990; Ourston and Mooney, 1990; Towell and Shavlik, 1991; Danyluk, 1991; Whitehall et al. 1991; Matwin and Plante, 1991 ] However, these systems are batch learners, which process all of the training instances at once. Knowledge assimilation requires the ability to incrementally revise a domain theory as new data is encountered. Incremental ....

....and Dietterich, 1989; Mooney and Ourston, 1989; Cohen, 1990 ] are not input output compatible since the output theory may not meet the required restrictions on the input theory. However, systems that can handle arbitrarily incorrect initial theories [ Ginsberg, 1990; Ourston and Mooney, 1990; Towell and Shavlik, 1991 ] can easily be made incremental. This paper presents empirical results on an incremental batch [ Clearwater et al. 1989 ] version of Either, a revision system for refining arbitrarily incorrect propositional Horn clause theories [ Ourston and Mooney, 1990; Mooney and Ourston, 1991b ] After ....

[Article contains additional citation context not shown here]

G. Towell and J. Shavlik. Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, pages 257--272, Harper's Ferry, W.Va., Nov. 1991.

....combinations of antecedents from the initial rules, which makes this a comparatively difficult learning task for a rule based system. Finally, it should be noted that when Kbann translated its results into Horn clauses, the resulting theory was significantly more complicated than Either s [Towell and Shavlik, 1991]. This is because Either s goal is to produce a minimally revised Horn clause theory and Kbann has no such bias. 7.2 Multiple categories: Soybean results In order to demonstrate Either s ability to revise multiple category theories, Either was used to refine the expert rules given in [Michalski ....

G. Towell and J. Shavlik. Refining symbolic knowledge using neural networks. In Proceedings of the International Workshop on Multistrategy Learning, pages 257--272, Harper's Ferry, W.Va., Nov. 1991.

No context found.

Towell G., Shavlik J. 1993. Refining Symbolic Knowledge Using Neural Networks, Machine Learning: An Artificial Intelligence Approach, vol. IV.

No context found.

Towell G., Shavlik J., Refining Symbolic Knowledge Using Neural Networks, Machine Learning: An Artificial Intelligence Approach, vol. IV, 1993.

No context found.

Towell, G. G. and Shavlik, J. W., "Refining Symbolic Knowledge Using Neural Networks," in Machine Learning: A Multistrategy Approach , Vol. IV, Michalski, R.S. and G. Tecuci, Morgan Kaufmann, San Mateo, CA, pp. 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC