G. G. Towell and J. W. Shavlik, "Extracting refined rules from knowledge-based neural networks," Machine Learning, vol. 13, pp. 71-- 101, 1993.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....model, they gave revision algorithms for threshold and parity functions. Goldsmith and Sloan [6] gave algorithms for 2 term monotone DNF in the general model and for propositional Horn sentences in the deletions only model. More generally, there is a wide AI literature on theory revision (e.g. [8,12,16]) Many systems for theory revision, such as EITHER [11] have been implemented. The problem of correcting errors is pervasive, and errorcorrecting algorithms appear in a variety of contexts. Among them are fault analysis of circuits in switching theory (see, e.g. Kohavi [7] program debugging ....

G. G. Towell and J. W. Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13:71--101, 1993.

....the decompositional approach is its transparency, since it relies on the analysis of the connections and weights Table 1. Literature relevant to rule extraction. Technique Reference Decompositional Category Fuzzy Pedagogical RuleNet McMillan et al. 23] # Subset Fu ( 24,25] Towell and Shavlik [26] # M of N Towell and Shavlik [26] Maire [27] # RULEX Andrews and Geva [28,29] # Similar weight Saito and Nakano [30] # VLA Thrun [31] # Extraction as learning Craven and Shavlik [32] # REFuNN Kasabov [33] # NEFCLASS Nauck and Kruse [34,35] # Fuzzy MLP Mitra [36] # FuNe I Halgamuge and ....

....transparency, since it relies on the analysis of the connections and weights Table 1. Literature relevant to rule extraction. Technique Reference Decompositional Category Fuzzy Pedagogical RuleNet McMillan et al. 23] # Subset Fu ( 24,25] Towell and Shavlik [26] # M of N Towell and Shavlik [26], Maire [27] # RULEX Andrews and Geva [28,29] # Similar weight Saito and Nakano [30] # VLA Thrun [31] # Extraction as learning Craven and Shavlik [32] # REFuNN Kasabov [33] # NEFCLASS Nauck and Kruse [34,35] # Fuzzy MLP Mitra [36] # FuNe I Halgamuge and Glesner [37] # of a trained neural ....

[Article contains additional citation context not shown here]

G. G. Towell and J.W. Shavlik, "Extracting refined rules from knowledge-based neural networks", Machine Learning, 13, pp. 71--101, 1993.

.... Processing Letters, 17(2) pp.149 164, 2003 It should be mentioned that the method proposed in this work is quite di#erent from the existing techniques for rule extraction from neural networks [14] For example, a wide class of existing methods extract symbolic rules from multilayer perceptrons [15]. Although fuzzy rule extraction has been studied in [2] the work is mainly based on a special feedforward neural network structure. In our work, fuzzy rules are extracted from RBF networks by investigating the di#erence between interpretable fuzzy rules and RBF networks. Since fuzzy rules and ....

G. Towell and J. Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13:71--101, 1993.

....practice [Calvin 1996] A lot of mixed systems have been essayed to cope the higher side of the coin, reasoning. We see neuro statistical, neuro genetic, neuro rule, neuro fuzzy, neuro chaos and the like. Also, recent algorithms for symbolic representation of neural networks [Gallant 1993] [Towell Shavlik 1993] help to bridge them. One of the great misconceptions in this area is to think that fuzzy and numerical approaches are just sufficient to fluid categories (e.g. assimilate a pen with a keyboard or the meaning of head in different contexts) We do not mean that fuzziness should be banned. ....

Towell, G.G.; Shavlik, J.W. "Extracting Refined Rules from Knowledge-Based Neural Networks" Machine Learning, Vol. 13, no. 1, pp. 71-101, Oct. 1993.

....networks (BP) to refine rulebased knowledge is the preservation of symbolic knowledge. Under the weight tuning process of a backpropagation algorithm, symbolic rules quickly lose their original meanings. In fact, large shifts in the meanings of hidden units can occur as a result of training [19]. Another major limitation of the BP approach is that the initial rule base has to be roughly complete, or else the initial network architecture created may not be sufficiently rich for dealing with the problem domain. As the standard backpropagation algorithm is not able to create additional ....

....on usage. Threshold pruning with threshold = 0:01 is applied, followed by the rule and antecedent pruning procedures using the local pruning strategy. Comparing predictive performance, rules extracted from Cascade ARTMAP are still slightly more accurate than the rules extracted from KBANN [18, 19]. While the Cascade ARTMAP rule sets contain more rules than the rule sets, the number of antecedents is almost half of that of the rule sets. The promoter rules formulated by Cascade ARTMAP are similar in form to the consensus sequences derived by conventional statistical methods. However, ....

[Article contains additional citation context not shown here]

G. G. Towell and J. W. Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13:71--101, 1993.

....of RE algorithms have been proposed [1] Andrews et al. 1995) introduced a classification scheme of RE algorithms comprising the following categories: RE technique, rule type, rule quality, RE complexity, and portability. We briefly describe these categories and classify the RE algorithms MofN [9] and VIA [10] whose details are presented below. The specific RE technique is either based on the internal structure of the net (decompositional) or the ANN is used to generate examples for a learning algorithm (pedagogical) MofN is decompositional, as it analyzes the incoming weights of each ....

....of the rule set, primarily based on rule set size, is another important rule quality factor. Both MofN and VIA exhibited classification accuracies close or equal to the trained ANN with high rule fidelity and could also outperform symbolic machine learning methods on certain benchmark problems [9, 8, 10]. The RE complexity of MofN can be estimated with the complexity of ANN training (as a retraining is necessary) VIA s complexity is dependent on its underlying linear optimization algorithm. The portability of an RE algorithm is an indicator for the generality of the specific technique. This is ....

[Article contains additional citation context not shown here]

Geoffrey G. Towell and Jude W. Shavlik. Extracting Refined Rules from Knowledge--Based Neural Networks. Machine Learning, 13:71--101, 1993.

....Through training with sample data, they can learn to classify data according to criteria not explicitly known beforehand. On the other hand, they are rather intractable and do not offer much insight into the decision process unless advanced approaches to find and extract the rules are used[9]. The computational effort can become quite high if several hidden layers are needed, but the calculations can be parallelized[10] An additional challenge is the fact that we are dealing with incomplete data. To our knowledge, not much has been published on the subject of missing data in the ....

Geoffrey G. Towell and Jude W. Shavlik, "Extracting refined rules from knowledge-based neural networks," Machine Learning, vol. 13, pp. 71--101, 1993.

....of other classification methods. We also know of no other work that extracts symbolic information from SVMs; hence, we are also motivated by the desire to use the high accuracy of SVMs to extract valuable symbolic features, similar to earlier works that did the same for multilayer perceptrons [14]. Our feature extraction process uses sensitivity analysis on the underlying SVM to identify a linear model and a single query modification that explains a subset of the desired documents. We extract additional query modifications by iterating the procedure with a new dataset composed of the ....

G. G. Towell and J. W. Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13:71, 1993.

....reduced, or when the average error decrease per epoch is too small, or when overfitting starts. E. Rule Refinement Several algorithms have been developed to extract propositional logic rules from trained NNs to obtain a symbolic description of the knowledge embedded in the network weights [3] [9], 10] 11] The quality of these rules depends on the network architecture and the performance of the network. If irrelevant parameters feature in the training set, or if a non optimal architecture with too many free parameters is used, the complexity of the rule generation process is increased ....

GG Towell, JW Shavlik, Extracting Refined Rules from Knowledge-Based Neural Networks, Machine Learning, Vol 13, 1993, pp 71-101

....cycles of activation (the initial symbolic knowledge for thematic roles can be seen in table 1) A sentence generator generates the input sentences. As soon as the training is over, symbolic rules can be obtained from the connectionist architecture by running an extraction procedure ( 3] 12] [13]) 3 Verb Microfeatures and Complementarity in HTRP The representations used by HTRP are based on McClelland and Kawamoto s [8] and Waltz and Pollack s [14] notion of semantic microfeature. For the verb, the representation is mainly derived from Franchi and Canado [2] Twenty binary semantic ....

Towell, G. G. and Shavlik, J. W.: Extracting Refined Rules from Knowledge-based Neural Networks, Machine Learning, 13 (1993) 71-101

....can now train the feed forward network corresponding to the program with knowledge not yet included in the program and given in form of data examples. Thereafter, a (hopefully) improved logic program can be extracted from the trained network using well known rule extraction techniques (see e.g. [11, 46, 2]) All results mentioned so far are propositional in nature. The logic programs are propositional ones, but also the rule extraction techniques mentioned in the previous paragraph generate only propositional rules. In fact, most of the research done so far and aiming at using neural networks for ....

G.G. Towell and J.W. Shavlik. Extracting Refined Rules from Knowledge-Based Neural Networks. Machine Learning, 13(1), pp. 71--101, 1993.

....architecture corresponds to the network learning and generalization capacities. As a consequence, the network is able to revise the initial symbolic rules. The fuzzy rule extraction from the network, after training, for both versions of HTRP is based on Fu [1993] Setiono and Liu [1996] and Towell and Shavlik [1993]. MACHINE LEARNING 856 For RIW, the final rules for the thematic role AGENT are the fo llowing: Hidden rules: If for verb ( 0.6 control of action) 1.0 direct process triggering) 0.1 direction to goal) 0.9 impacting process) 1.1 change of state) 0.1 no psychological state) ....

G. Towell and J. W. Shavlik. Extracting Refined Rules from Knowledge-Based Neural Networks. Machine Learning, 13, 71-101, 1993.

.... models are difficult to interpret i.e. it is not clear how the models arrive at the prediction or classification of a given input pattern [49] A number of people have considered the extraction of symbolic knowledge from trained neural networks for both feedforward and recurrent neural networks [12, 44, 57, 44]. For recurrent networks, the ordered triple of a discrete Markov process (fstate; input next stateg) can be extracted and used to form 16 46 48 50 52 54 56 58 60 0 10 20 30 40 50 60 70 80 90 Reject Percentage Reject Performance RNN Randomized Test Figure 9. Classification ....

Geoffrey G. Towell and Jude W. Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13:71, 1993.

....representation of the initial formulae, in our case we code them into bitstrings. However, unlike in KBANN, our representation has a clear and simple semantics, allowing us to translate the final theory back to an equivalent logical formula. It is possible to extract symbolic rules also from KBANN [115], but the extracted theory is only an approximation of the theory encoded in KBANN. In GAs relevant substructures in the background knowledge can be taken into consideration by filtering them through some mechanism and then introducing the result as chromosomes in the population. This approach ....

G. Towell and J. Shavlik. Extracting refined rules from knowledgebased neural networks. Machine Learning, 13(1):71--101, 1993.

....The number of rules equals the number of F a 2 nodes that become active during learning. the weight tuning process of a backpropagation algorithm, symbolic rules quickly lose their original meanings. In fact, large shifts in the meanings of hidden units can occur as a result of training [19]. Another major limitation of the BP approach is that the initial rule base has to be roughly complete, or else the initial network architecture created may not be sufficiently rich for dealing with the problem domain. As the standard backpropagation algorithm is not able to create additional ....

....usage. Threshold pruning with threshold =0:01 is applied, followed by the rule and antecedent pruning procedures using the local pruning strategy. Comparing predictive performance, rules extracted from Cascade ARTMAP are still slightly more accurate than the NofM rules extracted from KBANN [18] [19]. While the Cascade ARTMAP rule sets contain more rules than the NofM rule sets, the number TABLE V Performance of fuzzy ARTMAP, Cascade ARTMAP, and Cascade ARTMAP rules on the promoter data set comparing with the symbolic learning algorithm ID 3, the KNN system, consensus sequence analysis, the ....

[Article contains additional citation context not shown here]

G. G. TowellandJ.W.Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13:71--101, 1993.

No context found.

G. G. Towell and J. W. Shavlik, "Extracting refined rules from knowledge-based neural networks," Machine Learning, vol. 13, pp. 71-- 101, 1993.

No context found.

G. Towell and J. Shavlik, "Extracting refined rules from knowledgebased neural networks," Mach. Learn., vol. 13, pp. 71--101, 1993.

No context found.

G. G. Towell and J. W. Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13:71--101, 1993.

No context found.

G. Towell, J. Shavlik, Extracting refined rules from knowledge-based neural networks, Mach.Learning 13 (1) (1993) 71--101.

No context found.

G. G. Towell and J. W. Shavlik, "Extracting refined rules from knowledge -based neural networks," Machine Learning, vol. 13, no. 1, pp. 71--101, 1993.

No context found.

G. G. Towell and J. W. Shavlik. Extracting refined rules from knowledge-based neural networks. Machine Learning, 13:71--101, 1993.

No context found.

G. Towell and J. Shavlik, "Extracting refined rules from knowledgebased neural networks," Mach. Learn., vol. 13, pp. 71--101, 1993.

No context found.

Towell G., Shavlik J. 1993. Extracting Refined Rules from Knowledge-Based Neural Networks, Machine Learning, vol. 131, pp. 71-101.

No context found.

Towell, G. and J. Shavlik (1993). Extracting refined rules from knowledge-based neural networks. Machine Learning 13(1), 71 101.

No context found.

G.G. Towell and J.W. Shavlik, "Extracting Refined Rules from Knowledge-based Neural Networks," Machine Learning, Vol. 13, No. 1, 1993, pp. 71--1013.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC