Perrone M. Putting it all together: methods for combining neural networks. Advances in Neural Information Processing Systems 6, 1994; 1188--1189  Home/Search   Document Not in Database   Summary   Related Articles   Check

....result more e ective for PND classi ers rather than direct MLP and PLD classi ers. PND good generalization capabilities can be interpreted from di erent points of view. Parallel Nonlinear Dichotomizers choose a class using a series of separated dichotomizers. As a committee of neural networks [17], they carry out a kind of voting [11] 6] distributed over the dichotomizers subtasks. However, PND use di erent classi ers working on di erent dichotomic problems, lowering error bias [11] in a way similar to di erent classi ers working on the same problem [21] moreover the same algorithm is ....

.... However, PND use di erent classi ers working on di erent dichotomic problems, lowering error bias [11] in a way similar to di erent classi ers working on the same problem [21] moreover the same algorithm is repeated many times as in homogeneous voting, leading to a reduction of variance [17] [12] Analyzing error backpropagation during learning we can see that PND dichotomizers learn in a more specialized way compared with MLP classi ers. Learning of each PND dichotomizer takes place independently from other dichotomies and speci cally devoted to its proper dichotomic task, while ....

M.P. Perrone. Putting it all together: Methods for combining neural networks. In Alspector J. Cowan J.D., Tesauro G., editor, Advances in Neural Information Processing Systems, volume 6, pages 11881189. Morgan Kauman, San Francisco, CA, 1994.

....consist of two phases: model generation and model combination. It is important to generate a set of models that are diverse in the sense that they make errors in different ways. On the other hand, the types of errors made by the model set directly determine the appropriate combining strategy [28]. When the errors are uncorrelated, the optimal approach is to take the majority vote (also referred to as plurality voting) However, when patterns exist in the errors of the learned models, a more elaborate combining scheme is necessary. The focus of this research is on model combination. A ....

....pattern in the errors committed, the 12 CHRISTOPHER MERZ errors are said to be uncorrelated. If distinct patterns occur in the errors, e.g. f i is particularly good at classifying class c, then the errors are said to be correlated. In the former case, a simple approach like PV is most effective [28]. In the latter case, a more complex combining scheme is needed. An effective combining strategy must be able to adjust for both situations. Sections 5.1 and 5.2 evaluate how SCANN handles these two scenarios. Finally, Section 5.4 discusses how the scaled space derived using Correspondence ....

Michael P. Perrone. Putting it all together: Methods for combining neural networks. In Jack D. Cowan, Gerald Tesauro, and Joshua Alspector, editors, Advances in Neural Information Processing Systems, volume 6, pages 1188--1189. Morgan Kaufmann Publishers, Inc., 1994.

....consistently performs as well or better than other combining techniques on a suite of data sets. 1 Introduction Combining the predictions of a set of learned models 1 to improve classification and regression estimates has been an area of much research in machine learning and neural networks ([Wol92, MP97, Per94, LT93, Bre96, Mei95, TT95]) The challenge of this problem is to decide which models to rely on for prediction and how much weight to give each. The goal of combining learned models is to obtain a more accurate prediction than can be obtained from any single source alone. 1 A learned model may be anything from a ....

Michael P. Perrone. Putting it all together: Methods for combining neural networks. In Jack D. Cowan, Gerald Tesauro, and Joshua Alspector, editors, Advances in Neural Information Processing Systems, volume 6, pages 1188--1189. Morgan Kaufmann Publishers, Inc., 1994.

....for one hypothesis over another, the two hypotheses could both be generated and then voted. There are many different ways of producing and voting the hypotheses, and this is a very active topic of research, particularly in the neural network community (Perrone, 1993; Perrone Cooper, 1993; Perrone, 1994). There are strong Bayesian justifications for voting as well (Buntine, 1990) We explored two methods for generating multiple hypotheses. The first is bootstrapping (Efron Tibshirani, 1993; Breiman, 1994) Many equally plausible decision trees can be constructed by the following procedure. ....

Perrone, M. P. (1994). Putting it all together: Methods for combining neural networks. In Cowan, J. D., Tesauro, G., & Alspector, J. (Eds.), Advances in Neural Information Processing Systems, Vol. 6, pp. 1188--1189. Morgan Kaufmann, San Francisco, CA.

....explain, several algorithms reduce only errors caused by variance. These algorithms generate multiple hypotheses (e.g. by using different initial random weights in a neural network or different training sets to induce multiple decision trees or rule sets) and then vote among these hypotheses (Perrone 1994, Breiman 1994) They cannot reduce bias errors because the predictions among the multiple hypotheses are correlated. However, both variance and bias errors can be reduced by voting among multiple hypotheses produced by different learning algorithms applied to the same problem, assuming the bias ....

Perrone, M. P. 1994. Putting it all together: Methods for combining neural networks. Pages 1188--1190 in: Advances in Neural Information Processing Systems, 6, J. D. Cowan, G. Tesauro, and J. Alspector, editors. Morgan Kaufmann, San Mateo, California.

.... improved by generating multiple hypotheses (e.g. as a result of different random seeds in neural network learning or different training sets in decision tree learning) and then voting these hypotheses (Hansen Salamon, 1990; LeBlanc Tibshirani, 1993; Perrone, 1993; Perrone Cooper, 1993; Perrone, 1994; Meir, 1994; Breiman, 1994) We call this homogeneous voting, because multiple runs of the same algorithm on the same learning problem are combined by voting. We will argue from our definition of bias that homogeneous voting can only reduce variance and not bias. Some papers have also reported ....

Perrone, M. P. (1994). Putting it all together: Methods for combining neural networks. In Cowan, J. D., Tesauro, G., & Alspector, J. (Eds.), Advances in Neural Information Processing Systems, 6, 1188--1189. Morgan Kaufmann, San Francisco, CA.

No context found.

Perrone M. Putting it all together: methods for combining neural networks. Advances in Neural Information Processing Systems 6, 1994; 1188--1189

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC