Abstract. Classifier committee learning methods generate multiple classifiers to form a committee by repeatedly applying a single base learning algorithm. The committee members vote to decide the final classification. Two such methods, Bagging and Boosting, have shown great success with decision tree learning. They create different classifiers by modifying the distribution of the training set. This paper studies a different approach: the Stochastic Attribute Selection Committee learning method with decision tree learning. It generates classifier committees by stochastically modifying the set of attributes but keeping the distribution of the training set unchanged. An empirical evaluation of a variant of this method, namely Sasc, in a representative collection of natural domains shows that the SASC method can significantly reduce the error rate of decision tree learning. On average Sasc is more accurate than Bagging and less accurate than Boosting, although a one-tailed sign-test fails to show that these differences are significant at a level of 0.05. In addition, it is
|
3215
|
C4.5: Programs for Machine Learning
– Quinlan
- 1993
|
|
2438
|
Classification and Regression Trees
– Breiman, Friedman, et al.
- 1984
|
|
2138
|
UCI Repository of Machine Learning Databases
– Merz, Murphy
- 1996
|
|
1453
|
Bagging Predictors
– Breiman
- 1996
|
|
1133
|
A decision-theoretic generalization of on-line learning and an application to boosting
– Freund, Schapire
- 1997
|
|
1004
|
Experiments with a new boosting algorithm
– Schapire
- 1996
|
|
483
|
Boosting the Margin: A New Explanation for the Effectiveness of Voting Methods
– Schapire, Freund, et al.
- 1997
|
|
453
|
The strength of weak learnability
– Schapire
- 1990
|
|
356
|
An empirical comparison of voting classification algorithms: Bagging, boosting and variants
– Bauer, Kohavi
- 1999
|
|
338
|
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
– Kohavi
- 1995
|
|
337
|
Solving multiclass learning problems via error-correcting output codes
– Dietterich, Bakiri
- 1995
|
|
298
|
Boosting a weak learning algorithm by majority
– Freund
- 1995
|
|
232
|
Bagging, boosting and C4.5
– Quinlan
- 1996
|
|
199
|
Arcing classifiers
– Breiman
- 1998
|
|
81
|
A Theory of Learning Classification Rules
– Buntine
- 1990
|
|
79
|
C.J.: UCI Repository of machine learning databases, http://www.ics.uci.edu/~mlearn /MLRepository.html
– Blake, Keogh, et al.
- 1998
|
|
63
|
Multiple decision trees
– Kwok, Carter
- 1990
|
|
62
|
Hybrid system for protein secondary structure prediction
– Zhang, Mesirov, et al.
|
|
40
|
Machine learning bias, statistical bias, and statistical variance of decision tree algorithms
– Dietterich, Kong
- 1995
|
|
37
|
Stacked generalization, Neural Networks 5
– Wolpert
- 1992
|
|
33
|
Option decision trees with majority votes
– Kohavi, Kunz
- 1997
|
|
26
|
Why does bagging work? a bayesian account and its implications
– Domingos
- 1997
|
|
24
|
Bagging predictors, Machine Learning 24
– Breiman
- 1996
|
|
23
|
Learning probabilistic relational concept descriptions
– Ali
- 1996
|
|
20
|
An empirical comparison of voting classi cation algorithms: bagging, boosting and variants
– Bauer, Kohavi
- 1999
|
|
18
|
Boosting the margin: a new explanation for the e ectiveness of voting methods
– Schapire, Freund, et al.
- 1998
|
|
18
|
Ensembles as a sequence of classifiers
– Asker, Maclin
- 1997
|
|
13
|
Naive Bayesian Classifier Committees
– Zheng
- 1998
|
|
7
|
T.G.: Machine learning research
– Dietterich
- 1997
|
|
6
|
A theory of learning classi cation rules
– Buntine
- 1990
|
|
4
|
eds): Working Notes of AAAI Workshop on Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms (available at http://www.cs.fit.edu/~imlm/papers.html
– Chan, Stolfo, et al.
- 1996
|
|
3
|
Arcing classifiers. Technical Report (available at: http://www.stat
– Breiman
- 1996
|
|
3
|
Multiple boosting: A combination of boosting and bagging
– Zheng, Webb
- 1998
|
|
2
|
expert-level performance on a science image analysis task by a system using combined artificial neural networks
– Cherkauer
- 1996
|
|
2
|
Integrating Boosting and stochastic attribute selection committees for further improving the performance of decision tree learning
– Zheng, Webb, et al.
- 1998
|
|
2
|
Idealized models of decision committee performance and their application to reduce committee error
– Webb
- 1998
|
|
1
|
Bagging predictors. Machine Learning 24 (1996a) 123-140. Breiman, L.: Arcing classifiers
– Breiman
|
|
1
|
Naive Bayesian classi er committees
– Zheng
- 1998
|
|
