Fisher D., and McKusick K. 1989. An empirical comparison of ID3 and backpropagation. In International Joint Conference on Artificial Intelligence (IJCAI-89), 788793.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... in both fields as evidenced by the sizes of the two leading conferences (IJCNN and IJCAI) And relatively recently, researchers from both sides of the fence have begun to carry out comparative work to try to establish the precise strengths and weaknesses of symbolic and subsymbolic mechanisms (e.g. [7, 8, 9]) Initial results seem to suggest that in the field of learning, symbolic and subsymbolic algorithms are of comparable performance overall. Of course the existence of such comparative studies is not enough to prevent extremists in one paradigm or the other making exaggerated claims about the ....

....22 other] target primaryhypothyroid] output primaryhypothyroid] learnererror 0.0] pair 6 : input 19 2.3 66 1.09 61 SVHC] target primaryhypothyroid] output primaryhypothyroid] learnererror 0. 0249] pair 7 : This performance is comparable to that reported in [9] for the same training set. 10 [target nothyroidcondition] output nothyroidcondition] learnererror 0.0019] pair 17 : input 140 33 1.07 31 other] target primaryhypothyroid] output primaryhypothyroid] learnererror 0.0001] 6 How should we view neural nets ....

Fisher, D. and McKusick, K. (1989). An empirical comparison of ID3 and back-propagation. Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (pp. 788-793). Morgan Kaufmann.

....logically equivalent sets of rules. When the description spaces are not too large, this can be done using the DIAV concept visualization method, outlined in a later section. Presented studies follow several other efforts to compare learning methods and paradigms. For example, Fisher and McKusick [3] compared ID3 and a neural net using backpropagation algorithm on the problems of learning diagnostic rules for thyroid diseases, soybean plant diseases, and a few artificial problems. The comparison was based on the performance accuracy of descriptions as applied to testing examples and the ....

Fisher, D.H. and McKusick, K.B. 1989. An Empirical Comparison of ID3 and Backpropagation. In Proceedings of IJCAI-89. Detroit. MI: Morgan Kaufmann.

.... Others have also sought to define a benchmark for machine learning [58, 62, 63, 66, 138] or connectionist [97] models and have remarked on the lack of scientific rigor in case studies [83, 96, 97] Many others have conducted case studies most of which suffer the limitations noted above [14, 43, 44, 53, 68, 87, 108, 119, 121, 122, 132, 137]. One exception to this rule is the Statlog project [88] conducted by many machine learning researchers from October 1990 to June 1993 under the ESPRIT program of the European community. With this notable exception, it is the scope and rigor (reproducibility) of this case study that distinguishes ....

....implementing them listed below. 3.1 Decision Trees Decision trees are perhaps the most widely studied inductive learning models in the machine learning community. The literature abounds with papers proposing new models or variations of existing models and case studies using decision trees ([14, 21, 22, 25, 30, 34, 22 40, 43, 49, 50, 51, 53, 89, 93, 98, 99, 100, 101, 102, 104, 105, 106, 107, 109, 110, 111, 112, 113, 114, 118, 120, 123, 126, 129, 130, 131, 133, 134, 136]) For this case study, we use decision tree software from Quinlan and Buntine. Quinlan introduces decision trees and illustrates the use of his C4.5 software for decision trees (c4.5tree) and production rules derived therefrom (c4.5rule) in [105] Several decision tree algorithms (cart, id3, c4, ....

[Article contains additional citation context not shown here]

Douglas Fisher and Kathleen McKusick (1989). An Empirical Comparison of ID3 and Back-propagation. Proceedings of the 11th International Joint Conference on Artificial Intelligence. Morgan Kaufmann, San Mateo, CA.788-793.

....cluster containing in some sense similar documents. Several clustering methods and techniques have been proposed so far, such as, for example, based on similarity measures [van Rijsbergen, 1979; Salton and McGill, 1983] neighborhoods [Voorhees, 1985] hierarchies [Lebowitz, 1987; Willett, 1988; Fisher and McKusick, 1989; Crawford et al. 1991; Tanaka et al. 1999] on matrix theory (diagonalisation, singular value decomposition [Deerwester et al. 1990] Retrieval is performed based on a cluster representative which may but need not be one of the cluster members. If the particular retrieval method used ....

Fisher, D.H., and McKusick, K.B. (1989). An empirical comparison of ID3 and back-propagation. Proceedings of the 11th International Joint Conference on Artificial Intelligence (IJCAI-89), Detroit, MI, 788-793.

....examples are more accurately classified. Combining Explanation Based and Neural Learning 3 While the performance of ANNs that adjust their connection weights has been impressive, there are some problems with this form of machine learning: 1) Training times are lengthy. Recent experiments [Fisher89, Mooney89b, Shavlik89b, Weiss89] indicate that while ANNs classify new examples slightly better than do symbolic learning algorithms (such as ID3 [Quinlan86] they require 100 to 1000 times as much training time. 2) The initial weights can greatly effect how well the concept is learned. Because training ANNs usually involves ....

D. H. Fisher and K. B. McKusick, "An Empirical Comparison of ID3 and Back-propagation," Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, Detroit, MI, August 1989.

.... representational shift, rule extraction from neural networks Running Head: Extracting Rules Submitted (8 91) to Machine Learning Towell Shavlik Extracting Rules 2 1 Introduction Artificial neural networks (ANNs) have proven to be a powerful and general technique for machine learning (Fisher McKusick, 1989; Shavlik et al. 1991; Weiss Kapouleas, 1989; Ng Lippmann, 1990) However, ANNs have several well known shortcomings; perhaps the most significant is that a trained ANN is essentially a black box. That is, determining exactly why an ANN makes a particular decision is all but impossible. ....

....are true) then . As suggested previously, this method arose because we noticed that rule sets discovered by the Subset method often contain N of M style concepts. Further support for this method comes from experiments that indicate neural networks are good at learning N of M concepts (Fisher McKusick, 1989) as well as experiments with a variant of ID3 that show a bias towards N of M style concepts is useful (Murphy Pazzani, 1991) Finally, note that purely conjunctive rules result if N = M , while a set of disjunctive rules results when N = 1; hence, using N of M rules does not restrict ....

Fisher, D. H. & McKusick, K. B. (1989). An empirical comparison of ID3 and backpropagation. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, (pp. 788--793), Detroit, MI.

....training parameters; however, we feel there are distinct advantages to including neural networks in our study. First, previous empirical studies have demonstrated that individual neural networks produce highly accurate classifiers that are sometimes more accurate than corresponding decision trees (Fisher McKusick, 1989; Mooney, Shavlik, Towell, Gove, 1989) Second, neural networks have been extensively applied across numerous domains (Arbib, 1995) Finally, by studying neural networks in addition to decision trees we can examine how Bagging and Boosting are influenced by the learning algorithm, giving further ....

Fisher, D., & McKusick, K. (1989). An empirical comparison of ID3 and back-propagation. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, pp. 788--793 Detroit, MI.

....oblique rule, pruning, discretization. 1 Introduction Neural networks have been widely applied to solve classification problems. Comparisons between neural networks and decision trees algorithms for these problems have shown that in general neural networks can produce better accuracy rates [3, 4, 16, 20]. Recent developments in algorithms that extract rules from neural networks have made neural network techniques even more attractive. The extracted rules allow one to explain the decision process of a neural network. It is not surprising that in the past few years great efforts 1 have been ....

D.H. Fisher and K.B. McKusick. "An empirical comparison of ID3 and back-propagation," in Proceedings of 11th Int. Joint Conf. on AI, (1989) 788--793.

....for an incremental approach to place the hyperplanes optimally as opposed to the search methods used in standard decision trees. 1 To appear in the Proceedings of IJCNN, Seattle, 1991 Recently there have been many empirical studies comparing the performance of Neural Networks and Decision Trees [10, 11, 12, 13]. One particularly interesting test bed for pattern classifiers is Speech Recognition. The relative performances of MLPs and Decision Trees have been evaluated on a speaker independent vowel recognition task in [11, 12] These studies show that while MLPs sometimes have a smaller classification ....

D. Fisher and K. McKusick, "An Empirical Comparison of ID3 and Back-propagation," in Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, 1989.

....system. More generally, this work contributes to the understanding of how symbolic and connectionist approaches to artificial intelligence can be profitably integrated. 1 Introduction Artificial neural networks (ANNs) have proven to be a powerful and general technique for machine learning (Fisher and McKusick, 1989; Shavlik et al. 1991) Hence, it is tempting to use ANNs in combination with other, more knowledge intensive, learning strategies. However, ANNs have several well known shortcomings. Perhaps the most significant of these shortcomings is that it is very difficult to determine why a trained ANN ....

....explicitly searches for antecedents of the form: if (N of these M antecedents are true) then : This approach was taken because the rule sets discovered by the Subset method often contain N of M style concepts. Furthermore, experiments indicate that ANNs are good at learning N of M concepts (Fisher and McKusick, 1989) and that searching for N of M concepts is a useful inductive bias (Murphy and Pazzani, 1991) Finally, note that purely conjunctive rules result if N = M , while a set of disjunctive rules results when N = 1; hence, using N of M rules does not restrict generality. The idea underlying NofM, an ....

Fisher, D. H. and McKusick, K. B., "An empirical comparison of ID3 and backpropagation ", Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, pp. 788--793, Detroit, MI, 1989.

....Undergrad InTomsGroup Yes No Duration = 60 Duration = 30 Duration = 90 Figure 5. Part of an ID3 Decision Tree for Predicting Meeting Duration. propagation (Rumelhart et al. 1986) Both these methods have previously been shown to be effective in learning from examples (see, for example, (Fisher and McKusick, 1989) (Mooney et al. 1989) ID3 ID3 (Quinlan, 1986) Quinlan, 1987) is an inductive algorithm which builds a decision tree for classification of objects. Given a set of objects with known classes and described by a fixed set of features, a decision tree is produced which predicts the class of a new ....

....Table 7. Selected Rules Learned by ID3 for Location. of epochs. Similarly, there are many variations of ID3 which may cope better with noisy data, overfitting of data, etc. Thus we cannot draw too much from the comparison of these learning methods. However, our initial results confirm those of (Fisher and McKusick, 1989) (Mooney et al. 1989) Backpropagation performs slightly better than ID3 on average with noisy data, but takes much longer to train. We are currently investigating the possibility of initializing the network from the hand coded rules or ID3 generated rules, to try to improve the speed and ....

Fisher, D. and McKusick, K. (1989). An empirical comparison of id3 and back-propagation. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence. Morgan Kaufmann.

....is known to be complex relative to the amount of data available for training, Fisher and Schlimmer (1988) and Schaffer (1993) suggest that unpruned decision trees may be more accurate than pruned ones. And if we know that attribute values are heavily affected by noise, two studies cited by Quinlan (Fisher and McKusick, 1989; Shavlik et al. 1991) suggest that neural networks may outperform decision trees. As Quinlan concludes, our understanding of the applicability of each method is increasing and, as it does, we increase our ability to make use of problem specific knowledge, when we have it. Often, however, prior ....

Fisher, D. & McKusick, K. (1989). An empirical comparison of ID3 and back-propagation. Proceedings of the Eleventh International Joint Conference on Artificial Intelligence.

....build a connectionist network from a domain theory, but the method is limited to domain theories expressed in propositional logic. ffl Empirically, BACKPROP requires significantly more presentations than symbolic algorithms. Two recent empirical studies, Mooney, Shavlik, Towell Gove, 1989] and [Fisher McKusick, 1989], observed that BACKPROP takes substantially longer than ID3 on the same task (this conclusion was consistent over the six tasks total studied in the two research projects) The former study concluded that BACKPROP needed one to two orders of magnitude more presentations than ID3 to achieve the ....

.... weights have a strong influence on both the learning speed and the generalizing ability of the resulting network [Towell, Shavlik Noordewier, 1990] Also, the number of intermediate nodes needed for a network to solve a given problem varies a great deal, and is generally not determinable [Fisher McKusick, 1989]. If too few are used, the network will not attain 100 accuracy on the problem; if too many are used, the network will memorize the inputs and not form useful features, and thus not generalize well to unseen inputs. It is possible to use a dynamic network architecture whereby more intermediate ....

Fisher, D. H., & McKusick, K.B. (1989). An empirical comparison of ID3 and backpropagation.

.... and learning to pronounce English text (Sejnowski Rosenberg, 1987) In addition to these practical successes, several empirical studies have concluded that neural networks provide performance comparable to, and in some cases, better than common symbolic learning algorithms (Atlas et al. 1989; Fisher McKusick, 1989; Mooney et al. 1989) A distinct advantage of symbolic learning algorithms, however, is that the concept representations they form are usually more easily understood by humans than the representations formed by neural networks. In this paper we describe and investigate an approach for extracting ....

Fisher, D. H. & McKusick, K. B. (1989). An empirical comparison of ID3 and back-propagation. In Proc. of the 11th IJCAI, (pp. 788--793), Detroit, MI.

....755 College Road East, Princeton, NJ, 08540. Email: towell learning.siemens.com. Please direct all correspondence to this address. Towell Shavlik Extracting Rules 2 1 Introduction Artificial neural networks (ANNs) have proven to be a powerful and general technique for machine learning (Fisher McKusick, 1989; Shavlik et al. 1991) However, ANNs have several well known shortcomings; perhaps the most significant of which is that a trained ANN is essentially a black box. That is, determining exactly why an ANN makes a particular decision is a daunting task. This is a significant shortcoming, for ....

....specified by the remaining links. 6. Where possible, simplify rules to eliminate superfluous weights and thresholds. ered by Subset often contain M of N style concepts. Further support for this method comes from experiments that indicate neural networks are good at learning M of N concepts (Fisher McKusick, 1989), as well as experiments with a variant of ID3 that show a bias towards M of N style concepts is useful (Murphy Pazzani, 1991) Finally, note that purely conjunctive rules result if N = M , while a set of disjunctive rules results when M = 1; hence, using M of N rules does not restrict ....

Fisher, D. H. & McKusick, K. B. (1989). An empirical comparison of ID3 and backpropagation.

....neural networks (ANNs) have proven to be a powerful and general technique for machine learning. For example, a host of empirical comparisons indicate that ANNs are at least as effective at generalizing from training to testing examples as any of several common symbolic machine learning algorithms [Atlas89, Fisher89, Shavlik91, Weiss89]. This generalization aptitude results from the ability of ANNs to train hidden units to form useful intermediate representations. In other words, the success of ANNs results from their ability to use hidden units as loci for constructive induction. However, ANNs tend to be black boxes after ....

....rule. Observation of trained KNNs suggests that both of these assumptions are valid. The NofM method searches for antecedents of the form: if (N of these M antecedents are true) then . NofM was suggested by experiments that indicate neural networks are good at learning N of M concepts [Fisher89] and our observation that rule sets extracted by methods similar to [Saito88] often contain subsets which can be compactly expressed as N of M concepts. The NofM method finds groups of links that have approximately the same weight and treats each group as if all of the weights in it were the ....

Fisher, D. H. and McKusick, K. B. (1989). An empirical comparison of ID3 and back-propagation. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, pages 788--793, Detroit, MI.

....several reasons why it is sometimes preferable to use neural networks for a concept learning task rather than a symbolic learning method. A major reason is that neural networks offer better generalization performance than common symbolic learning algorithms for some problems (Atlas et al. 1989; Fisher McKusick, 1989; Weiss Kapouleas, 1989; Shavlik et al. 1991) When putting learning systems into practice, generalization performance is usually the overriding criterion for selecting one algorithm over another. Another reason for preferring neural networks over symbolic algorithms is that they can be applied ....

Fisher, D. H. & McKusick, K. B. (1989). An empirical comparison of ID3 and back-propagation. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, (pp. 788--793), Detroit, MI.

....gained much popularity in business applications as well as in computer science, psychology and cognitive science. Neural nets have been successfully applied to loan evaluation, signature recognition, time series forecasting (Dutta and Shekhar 1988; Sharda and Patil 1990) classification analysis (Fisher and McKusick 1989; Singleton and Surkan 1990) and many other difficult pattern recognition problems (Simpson 1990) While it is often difficult or impossible to explicitly write down a set of rules for such pattern recognition problems, a neural network can be trained with raw data to produce a solution. ....

Fisher, D. H. and McKusick, K. B. 1989. An empirical comparison of id3 and back-propagation.

....is that empirical results have illustrated that neural nets, trained using the BACKPROP algorithm, require significantly more training instances than symbolic algorithms to converge to a satisfactory generalization. Studies have shown that BACKPROP takes much longer than ID3 on the same tasks (Fisher McKusick, 1989; Mooney, Shavlik, Towell Gove, 1989) For these test cases the error rates of ID3 and BACKPROP were not significantly different. When noise was added to the test cases, BACKPROP outperformed ID3. However, it is unclear how much of this difference should be attributed to the pruning methods used ....

Fisher, D. H., & McKusick, K.B. (1989). An empirical comparison of ID3 and backpropagation.

.... 1990; Gorman and Sejnowski, 1988; Thrun et al. 1991; Atlas et al. 1990) but the results were inconclusive since the data sets were too small and selective, or little analysis was done (Thrun et al. 1991) Several more extensive studies have been published, such as (Weiss and Kulikowski, 1991; Fisher and McKusick, 1989; Shavlik, Mooney, and Towell, 1991; Feng et al. 1992; Ripley, 1992) Shavlik, Mooney and Towell (1991) carried out a thorough experimental comparison between ID3, ANNs using BP and the perceptron on five real world data sets including the famous connectionist text to speech conversion model ....

Fisher, D. & McKusick, K. (1989). An empirical comparison of ID3 and back-propagation.

No context found.

Fisher D., and McKusick K. 1989. An empirical comparison of ID3 and backpropagation. In International Joint Conference on Artificial Intelligence (IJCAI-89), 788793.

No context found.

D. H. Fisher and K. B. McKusick. 1989. An empirical comparison of id3 and back-propagation (vol 1). IJCAI 89, pp 788 -- 793, San Mateo, CA. Morgan Kaufmann.

No context found.

Fisher, and McKusik. 1989. An empirical comparison of ID3 and back propagation. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, 788-793.

No context found.

Fisher, D. H. and McKusick, K. B. (1989). An empirical comparison of ID3 and backpropagation.

No context found.

Fisher, D., and McKusick, K. (1989) An empirical comparison of ID3 and back-propagation. Proceedings of the International Joint Conference on Artificial Intelligence, 788-793. San Mateo, CA: Morgan Kaufmann.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC