Abstract:
AdaBoost is a method for incrementally creating a classier ensemble. We investigate how the diversity of an ensemble of classiers created by AdaBoost varies as the number of classiers in the ensemble increases. We consider two data sets from the UCI machine learning repository and use ten dierent measures of diversity. We show that AdaBoost does indeed initially increase the diversity but after the rst few classiers the diversity begins to gradually tail o. These results suggest that useful classier ensembles can be recovered at an early stage of AdaBoost training, perhaps using a more sophisticated combination method than the weighted voting.
Citations
|
1453
|
Bagging Predictors
– Breiman
- 1996
|
|
368
|
Neural network ensembles
– Hansen, Salamon
- 1990
|
|
347
|
Statistical pattern recognition: A review
– Jain, Duin, et al.
- 2000
|
|
134
|
Bias plus variance decomposition for zero-one loss functions
– Kohavi, Wolpert
- 1996
|
|
102
|
Statistical methods for rates and proportions
– Fleiss
- 1981
|
|
40
|
An empirical comparison of voting classi algorithms: Bagging, boosting, and variants
– Bauer, Kohavi
- 1999
|
|
37
|
Theoretical Views of Boosting
– Schapire
- 1999
|
|
22
|
Classifier combinations: implementations and theoretical issues
– Lam
- 2000
|
|
17
|
The sources of increased accuracy for two proposed boosting algorithms
– Skalak
- 1996
|
|
16
|
Software diversity: practical statistics for its measurement and exploitation
– Partridge, Krzanowski
- 1997
|
|
13
|
On the association of attributes in statistics
– Yule
- 1900
|
|
11
|
The random space method for constructing decision forests
– Ho
- 1998
|
|
9
|
Distinct failure diversity in multiversion software. (personal communication
– Partridge, Krzanowski
- 1999
|
|
7
|
Design of eective neural network ensembles for image classi processes
– Giacinto, Roli
- 2000
|
|
7
|
Treating harmful collinearity in neural network ensembles
– Hashem
- 1999
|
|
7
|
Is Independence Good for Combining Classi
– Kuncheva, Whitaker, et al.
- 2000
|
|
6
|
Ten measures of diversity in classi ensembles: limits for two classi
– Kuncheva, Whitaker
- 2001
|
|
5
|
Performance degradation in boosting
– Wickramaratna, Holden, et al.
- 2001
|
|
4
|
Measures of diversity in classi ensembles. (submitted
– Kuncheva, Whitaker
|
|
3
|
Boosting neural networks
– Drucker
- 1999
|
|
1
|
Pattern Classi chapter 9
– Duda, Hart, et al.
- 2001
|
|
