Abstract

Recent work has shown that combining multiple versions of unstable classifiers such as trees or neural nets results in reduced test set error. One of the more effective is bagging (Breiman [1996a] ) Here, modified training sets are formed by resampling from the original training set, classifiers constructed using these training sets and then combined by voting. Freund and Schapire [1995,1996] propose an algorithm the basis of which is to adaptively resample and combine (hence the acronym--arcing) so that the weights in the resampling are increased for those cases most often misclassified and the combining is done by weighted voting. Arcing is more successful than bagging in test set error reduction. We explore two arcing algorithms, compare them to each other and to bagging, and try to understand how arcing works. We introduce the definitions of bias and variance for a classifier as components of the test set error. Unstable classifiers can have low bias on a large range of data sets....

Citations

1714	Schapire R: A decision-theoretic generalization of online learning and an application to boosting - Freund - 1997
144	Bias plus variance decomposition for zero-one loss functions - Kohavi, Wolpert - 1996
81	Boosting decision trees - Drucker, H, et al. - 1996
75	Neural networks and the bias/variance dilemma. Neural Comput - Geman, Bienenstock, et al. - 1992
13	1996] Experiments with a new boosting algorithm - Freund, Schapire
4	Learning Probablistic Relational Concept Descriptions - Ali - 1995