R. Maclin, J.S.: Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks. In: Proceedings of IJCAI-95. (1995) 524--530  Home/Search   Document Details and Download   Summary   Related Articles   Check

....many other approaches for training individual neural networks. Examples are as follows. Hampshire and Waibel [15] utilize di#erent objective functions to train di#erent individual neural networks. Cherkauer [6] trains individual networks with di#erent number of hidden units. Maclin and Shavlik [25] initialize individual networks at di#erent points of the weight space. Krogh and Vedelsby [23] employ cross validation to create individual networks. Opitz and Shavlik [28] exploit genetic algorithm to train diverse knowledge based neural networks. Yao and Liu [46] ensemble all the individuals in ....

R. Maclin and J. W. Shavlik, Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks, in: Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, Canada, 1995, pp.524-530.

....assume that the classifiers forming the classifier ensemble have to be both diverse and accurate. The diversity assumption means that the classifiers have to make independent classification errors, in order to improve overall prediction accuracy. Both theoretical [7, 8] and empirical work [9, 10] has shown that a good neural network ensemble is one where the individual networks are both accurate and make errors on different parts of the input space. For example, Hansen and Salamon [7] have shown that a multiple classifier system based on a simple majority combination rule can provide very ....

Maclin, R. and Shavlik, J., Combining the Predictions of Multiple Classifiers: Using Competitive Learning to Initialize Neural Networks,Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1995.

.... accuracy [18] Numerous previous works on neural networks committees have shown that an efficient committee should consist of networks that are not only very accurate but also diverse in the sense that the networks make their independent errors in different regions of the input space [21, 22]. For a instance, the combination of two neural networks that agree everywhere cannot lead to any accuracy improvement, no matter how ingenious a combination method is employed. It has been recently shown that the half half bagging through the majority voting rule is capable of creating very ....

Maclin, R., J.W. Shavlik,. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks, Proceedings of the 14 th International Conference on Artificial Intelligence. 1995.

....if more than half of the individual networks vote to the prediction. Plurality voting judges a prediction to be the final output if the prediction ranks first according to the number of votes. Much work has been done in designing ensemble approaches. Examples are as follows. Maclin and Shavlik [18] utilized competitive learning to generate individual networks and then combined their outputs via simple averaging. Taniguchi and Tresp [28] designed variance based weighting and variance based Bagging, and experimentally found that the improvement in performance that could be achieved by ....

Maclin R, Shavlik JW. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1995. p.524-530.

....likely to make errors in different ways. Plurality voting with a model set consisting of a neural networks, decision trees, rule sets, and other models was shown to be effective in [22] The search strategy of a learning algorithm may also be modified to diversify the model set. Maclin and Shavlik [19] accomplished this by strategically initializing the weights of a neural network. Ali and Pazzani [1] generated decision lists (i.e. list of rules) where conditions are added to a rule stochastically. As with the first approach, the models are typically combined using variants of the weighted ....

.... was referred earlier as the plurality vote (PV) and is also known as the basic ensemble method (BEM) 27] This approach has frequently been used as a straw man combining scheme for comparing to other combining schemes [22] or as a simple combining scheme to evaluate model generation strategies [3, 19]. A more elaborate weighting scheme derived by Perrone and Cooper [27] is the general ensemble method (GEM) GEM is different from SCANN in that models are assigned fixed weights, and GEM has difficultly dealing with models that make highly correlated errors. One can also combine learned models ....

R. Maclin and J. W. Shavlik. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1995. 26 CHRISTOPHER MERZ

....predictions. For example, neural network techniques that have been employed include methods for training with di#erent topologies, di#erent initial weights, di#erent parameters, and training only on a portion of the training set (Alpaydin 1993; Freund Schapire 1996; Hansen Salamon 1990; Maclin Shavlik 1995). Numerous techniques try to generate disagreement among the classifiers by altering the training set each classifier sees. The two most popular techniques are Bagging (Breiman 1996) and Boosting (Freund Schapire 1996) Bagging is a bootstrap ensemble method that trains each network in the ....

Maclin, R., and Shavlik, J. 1995. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence.

.... combining classifiers can lead to increased generalization accuracy for a variety of classifier model classes and domains (e.g. Selfridge 1959; Wolpert 1992; Chan and Stolfo 1995; Breiman 1992; Breiman 1994a; Kong and Dietterich 1995; Ali and Pazzani 1995; Zhang et al. 1992; Jacobs et al. 1991; Maclin and Shavlik 1995; Skalak 1995) In particular, the accuracy of some classifiers can be boosted by combining their predictions with those of other classifiers (Schapire 1990; Drucker et al. 1994; Drucker and Cortes 1996; Freund and Schapire 1995) A logical next research task is to identify the sources of ....

Maclin, R. and Shavlik, J.W. 1995. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Mellish, C.S., editor, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence. Morgan Kaufmann, San Mateo, CA. 524--530.

....is an incremental technique that is used 4 Personal communication. 46 to find a given number of average instances. Beginning with a set of random instance seeds, simple competitive learning moves a seed closer to a training instance each time a seed is the closest one [Hertz et al. 1991; Maclin and Shavlik, 1995] The number of (input, output and hidden) nodes are parameters for many neural network algorithms. A similar issue arises in pruning a decision tree. Determining the number of nodes in a decision tree or the number of computational units in a neural network may be more a matter of architecture ....

....method. The reliance on these two techniques may be surprising in view of their shortcomings. 2.4.1 Function Approximation 2.4.1. 1 Linear Where classifiers output a real value prediction, a linear combination function is a common combiner [Breiman, 1992; Hashem, 1993; Oliver and Hand, 1995; Maclin and Shavlik, 1995; Tumer and Ghosh, 1996] The linear combination may be simple, unweighted averaging or a weighted combination of predictions. Simple linear regression algorithms are often used to compute weights on the component classifier predictions. One advantage of linear combination methods is that it does ....

Maclin, R. and Shavlik, J.W. 1995. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Mellish, C.S., editor, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence. Morgan Kaufmann, San Mateo, CA. 524--530.

....feed forward network, which comprises of one input layer, one or more middle, or hidden layers, and one output layer. There are some other types of networks, such as functional link networks, counterpropagation networks, and adaptive response theory networks, etc. which are introduced in [52] [39] proposed an interesting method, called combining neural network, which trains a number of networks and then somehow uses the collection to increase generalization. 34] proposed a weighted effect approach using recursive neural nets. This approach has the flexibility of predicting number of ....

R. Maclin and J. W. Shavlik. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proc. of the 14th Int. Joint Conf. on Artificial Intelligence (IJCAI-95), pages 524--530, Montreal, Canada, August 1995.

....generally focuses (if only indirectly) on the selection of component classifiers that are accurate and differ in their predictions. Examples of mechanisms that have been used to create component classifiers include using different training parameters with a single learning method (Alpaydin 1993; Maclin Shavlik 1995), using different subsets of the training data with a single learning method (Breiman 1996a; Freund Schapire 1996) using different learning methods (Zhang, Mesirov, Waltz 1992) and explicitly searching for a set of classifiers that is both accurate and diverse (Opitz Shavlik 1996) ....

Maclin, R., and Shavlik, J. 1995. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In IJCAI-95, 524--530.

.... classification methods, training on subsets of the data set, training on different sets of input features, and using different subsets of the training set for training the classifiers [ Breiman, 1996; Drucker et al. 1994; Hansen and Salamon, 1990; Hashem et al. 1994; Krogh and Vedelsby, 1995; Maclin and Shavlik, 1995 ] In the present application, Sequel combined classifiers that were trained using different input features (see Section 6) The second aspect of creating an ensemble is the choice of the function for combining the predictions of the component classifiers [ Kearns and Seung, 1995 ] Examples ....

R. Maclin and J. Shavlik. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Montreal, Canada, September 1995.

.... of input features, using different subsets of the training set for training the classifiers, and even using genetic search to try to find classifiers that disagree in their predictions (Breiman 1996; Drucker et al. 1994; Hansen Salamon 1990; Hashem, Schmeiser, Yih 1994; Krogh Vedelsby 1995; Maclin Shavlik 1995; Opitz Shavlik 1996) The method of choosing different classification methods is interesting, since most classification methods introduce particular biases into the resulting classification. Also appealing is the method of varying the set of input features, since the resulting component ....

Maclin, R., and Shavlik, J. 1995. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence.

....predictions. For example, neural network techniques that have been employed include methods for training with different topologies, different initial weights, different parameters, and training only on a portion of the training set (Alpaydin 1993; Drucker et al. 1994; Hansen Salamon 1990; Maclin Shavlik 1995). In this paper we concentrate on two popular methods (Bagging and Boosting) that try to generate disagreement among the classifiers by altering the training set each classifier sees. Bagging Classifiers Bagging (Breiman 1996a) is a bootstrap (Efron Tibshirani 1993) ensemble method that ....

Maclin, R., and Shavlik, J. 1995. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 524--530.

.... 1992; Zhang et al. 1992) In particular, combining separately trained neural networks (commonly referred to as a neural network ensemble) has been demonstrated to be particularly successful (Alpaydin, 1993; Drucker et al. 1994; Hansen Salamon, 1990; Hashem et al. 1994; Krogh Vedelsby, 1995; Maclin Shavlik, 1995; Perrone, 1992) Both theoretical (Hansen Salamon, 1990; Krogh Vedelsby, 1995) and empirical (Hashem et al. 1994; Maclin Shavlik, 1995) work has shown that a good ensemble is one where the individual networks are both accurate and make their errors on different parts of the input space; ....

.... been demonstrated to be particularly successful (Alpaydin, 1993; Drucker et al. 1994; Hansen Salamon, 1990; Hashem et al. 1994; Krogh Vedelsby, 1995; Maclin Shavlik, 1995; Perrone, 1992) Both theoretical (Hansen Salamon, 1990; Krogh Vedelsby, 1995) and empirical (Hashem et al. 1994; Maclin Shavlik, 1995) work has shown that a good ensemble is one where the individual networks are both accurate and make their errors on different parts of the input space; however, most previous work has either focussed on combining the output of multiple trained networks or only indirectly addressed how one should ....

[Article contains additional citation context not shown here]

Maclin, R. & Shavlik, J. (1995). Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Montreal, Canada.

.... 1989; Wolpert, 1992) In particular, combining separately trained neural networks (commonly referred to as a neural network ensemble) has been demonstrated to be particularly successful (Alpaydin, 1993; Drucker et al. 1994; Hansen and Salamon, 1990; Hashem et al. 1994; Krogh and Vedelsby, 1995; Maclin and Shavlik, 1995; Perrone, 1992) Both theoretical (Hansen and Salamon, 1990; Krogh and Vedelsby, 1995) and empirical (Hashem et al. 1994; Maclin and Shavlik, 1995) work has shown that a good ensemble is one where the individual networks are both accurate and make their errors on different parts of the input ....

.... to be particularly successful (Alpaydin, 1993; Drucker et al. 1994; Hansen and Salamon, 1990; Hashem et al. 1994; Krogh and Vedelsby, 1995; Maclin and Shavlik, 1995; Perrone, 1992) Both theoretical (Hansen and Salamon, 1990; Krogh and Vedelsby, 1995) and empirical (Hashem et al. 1994; Maclin and Shavlik, 1995) work has shown that a good ensemble is one where the individual networks are both accurate and make their errors on different parts of the input space; however, most previous work has either focussed on combining the output of multiple trained networks or only indirectly addressed how we should ....

Maclin, R. and Shavlik, J. (1995). Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Montreal, Canada.

No context found.

R. Maclin, J.S.: Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks. In: Proceedings of IJCAI-95. (1995) 524--530

No context found.

R. Maclin and J. W. Shavlik, Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks, in: Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, Canada, 1995, pp.524-530.

No context found.

R. Maclin, J. W. Shavlik, Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks, in: Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, Canada, 1995, pp. 524--530.

No context found.

R. Maclin and J. W. Shavlik. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1995.

No context found.

R. Maclin and J. W. Shavlik, Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks, in: Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, Canada, 1995, pp.524-530.

No context found.

R. Maclin and J. W. Shavlik. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1995.

No context found.

R. Maclin, J.S.: Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks. In: Proceedings of IJCAI-95. (1995) 524--530

No context found.

R. Maclin, J.W. Shavlik. Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. Proceedings of the 14 th International Conference on Artificial Intelligence, 1995.

No context found.

R. Maclin and J. W. Shavlik, Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks, in Proc. 14th Int. Joint Conf. on Artificial Intelligence, Montreal, Canada, 1995, 524-530.

No context found.

Maclin, R., and Shavlik, J., Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Montreal, Canada, 1995.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC