Abstract. Classifier committee learning methods generate multiple classifiers to form a committee by repeated application of a single base learning algorithm. The committee members vote to decide the final classification. Two such methods, Bagging and Boosting, have shown great success with decision tree learning. They create different classifiers by modifying the distribution of the training set. This paper studies a different approach: Stochastic Attribute Selection Committee learning of decision trees. It generates classifier committees by stochastically modifying the set of attributes but keeping the distribution of the training set unchanged. An empirical evaluation of a variant of this method, namely Sasc, in a representative collection of natural domains shows that the SASC method can significantly reduce the error rate of decision tree learning. On average Sasc is more accurate than Bagging and less accurate than Boosting, although a one-tailed sign-test fails to show that these differences are significant at a level of 0.05. In addition, it is
|
3248
|
C4.5: programs for machine learning
– Quinlan
- 1993
|
|
2475
|
Classification and Regression Trees
– Breiman, Friedman, et al.
- 1984
|
|
2163
|
UCI Repository of machine learning databases
– Blake, Merz
- 1998
|
|
1470
|
Bagging predictors
– Breiman
- 1996
|
|
1149
|
A decision-theoretic generalization of on-line learning and an application to boosting
– Freund, E
- 1997
|
|
1015
|
Experiments with a New Boosting Algorithm
– FREUND, SCHAPIRE
|
|
489
|
Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics
– Schapire, Freund, et al.
- 1998
|
|
460
|
The strength of weak learnability
– Schapire
- 1987
|
|
358
|
An empirical comparison of voting classification algorithms: Bagging, boosting, and variants
– Bauer, Kohavi
- 1999
|
|
341
|
Solving Multiclass Learning Problems via Error-Correcting Output Codes
– Dietterich, Bakiri
- 1995
|
|
340
|
A study of cross-validation and bootstrap for accuracy estimation and model selection
– Kohavi
- 1995
|
|
302
|
Boosting a weak learning algorithm by majority
– FREUND
|
|
233
|
Bagging, boosting, and C4.5
– Quinlan
- 1996
|
|
199
|
Arcing classifiers
– Breiman
- 1998
|
|
82
|
A Theory of Learning Classification Rules
– Buntine
- 1990
|
|
79
|
C.J.: UCI Repository of machine learning databases, http://www.ics.uci.edu/~mlearn /MLRepository.html
– Blake, Keogh, et al.
- 1998
|
|
64
|
Multiple decision trees
– Kwok, Carter
- 1990
|
|
62
|
Hybrid system for protein secondary structure prediction
– Zhang, Mesirov, et al.
|
|
40
|
Machine learning bias, statistical bias, and statistical variance of decision tree algorithms
– Dietterich, Kong
- 1995
|
|
37
|
Stacked generalization, Neural Networks 5
– Wolpert
- 1992
|
|
33
|
Option decision trees with majority votes
– Kohavi, Kunz
- 1997
|
|
26
|
does bagging work? A Bayesian account and its implications
– Domingos, Why
- 1997
|
|
24
|
Bagging predictors, Machine Learning 24
– Breiman
- 1996
|
|
23
|
Learning probabilistic relational concept descriptions
– Ali
- 1996
|
|
20
|
An empirical comparison of voting classi cation algorithms: Bagging, boosting, and variants
– Bauer, Kohavi
|
|
18
|
Boosting the margin: A new explanation for the e ectiveness of voting methods
– Schapire, Freund, et al.
- 1998
|
|
18
|
Ensembles as a sequence of classifiers
– Asker, Maclin
- 1997
|
|
13
|
Naive Bayesian Classifier Committees
– Zheng
- 1998
|
|
7
|
T.G.: Machine learning research
– Dietterich
- 1997
|
|
6
|
A theory of learning classi cation rules
– Buntine
- 1990
|
|
4
|
eds): Working Notes of AAAI Workshop on Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms (available at http://www.cs.fit.edu/~imlm/papers.html
– Chan, Stolfo, et al.
- 1996
|
|
3
|
Arcing classifiers. Technical Report (available at: http://www.stat
– Breiman
- 1996
|
|
3
|
Multiple boosting: A combination of boosting and bagging
– Zheng, Webb
- 1998
|
|
2
|
expert-level performance on a science image analysis task by a system using combined artificial neural networks
– Cherkauer
- 1996
|
|
2
|
Integrating Boosting and stochastic attribute selection committees for further improving the performance of decision tree learning
– Zheng, Webb, et al.
- 1998
|
|
2
|
Idealized models of decision committee performance and their application to reduce committee error
– Webb
- 1998
|
|
1
|
Bagging predictors. Machine Learning 24 (1996a) 123-140. Breiman, L.: Arcing classifiers
– Breiman
|
|
1
|
Naive Bayesian classi er committees
– Zheng
- 1998
|
|
