Citation Context

...A variety of other examples (not shown) exhibit similar behavior with all boosting methods. Note that other committee approaches to classification such as bagging (Breiman 1996) and randomized trees (=-=Dietterich 1998-=-), while admitting parallel implementations, cannot take advantage of this approach to reduce computation. 9 Concluding remarks In order to understand a learning procedure statistically it is necessar...

Citation Context

...istent with theory, boosting can fail to perform well given insu cient data, overly complex weak hypotheses or weak hypotheses which are too weak. Boosting seems to be especially susceptible to noise =-=[10]-=-. AdaBoost has been tested empirically by many researchers, including [2, 10, 12, 21, 25, 28, 36]. For instance, Freund and Schapire [16] tested AdaBoost on a set of UCI benchmark datasets [27] using ...

Citation Context

...tiple random initial weights third best on one synthetic data set and two medical diagnosis data sets. For the C4.5 decision tree algorithm, it is also easy to inject randomness (Kwok & Carter, 1990; =-=Dietterich, 2000-=-). The key decision of C4.5 is to choose a feature to test at each internal node in the decision tree. At each internal node, C4.5 applies a criterion known as the information gain ratio to rank-order...

Citation Context

...r, uncorrelated classifiers reduce the added error by a factor of 1/L , and negatively correlated classifiers reduce the error even further. But note that there is a limit on the largest absolute value of a negative pairwise correlation among L classifiers. Tumer and Ghosh (1996b, 1999) do not mention the case of negative correlation although it clearly supports their thesis that the smaller the correlation, the better the ensemble. A negative correlation between the continuousvalued outputs has been sought, predominantly by altering the available training set or parameters of the classifier (Dietterich, 2000a; Hashem, 1999; Krogh & Vedelsby, 1995; Liu & Yao, 1999; Opitz & Shavlik, 1999; Parmanto et al., 1996; Giacinto & Roli, 2001; Rosen, 1996; Sharkey & Sharkey, 1997; Skalak, 1996; Tumer & Ghosh, 1999). – When classifiers output class labels, the classification error can be decomposed into bias and variance terms (also called ‘spread’) (Bauer & Kohavi, 1999; Breiman, 1999; Kohavi & Wolpert, 1996) or into bias and spread terms. In both cases the second term can be taken as the diversity of the ensemble. These results have been used to study the behavior of classifier ensembles in terms of the bia...

Citation Context

...operational code. As a result, C4.5 is being used because of efficiency considerations and because of its ability to handle missing data. Friedl, Brodley, and Strahler (1999), Friedl et al. (2000a,b) and Gopal, Woodcock, and Strahler (1999) provide details regarding the evaluation and testing of C4.5 and ARTMAP for the MLCCA. A key feature of the MLCCA is a technique known as ‘‘boosting’’ (Freund, 1995). Boosting is one of numerous ensemble classification methods that were developed in the mid- to late 1990s that have been widely shown to enhance classification accuracy (Bauer & Kohavi, 1999; Dietterich, 2000). Boosting also serves to minimize the sensitivity of the classification algorithm to noise in feature data and labeling errors in training data. While many forms of ensemble algorithms have been developed, most use iterative methods where multiple classifications are estimated based on resampled versions of the training data. The final classification for each case is then produced by a weighted vote. Indeed, Friedman, Hastie, and Tibshirani (2000) recently described boosting as ‘‘one of the most important recent developments in classification methodology.’’ While most of the development work ...