Download:
by Zhi-hua Zhou, Jian-xin Wu, Wei Tang, Zhao-qian Chen
http://cs.nju.edu.cn/people/zhouzh/zhouzh.files/Publication/ijcia.pdf
Add To MetaCart
Abstract:
Neural network ensemble is a learning paradigm where a collection of neural networks is trained for the same task. In this paper, the relationship between the generalization ability of the neural network ensemble and the correlation of the individual neural networks constituting the ensemble is analyzed in the context of combining neural regression estimators, which reveals that ensembling a selective subset of trained networks is superior to ensembling all the trained networks in some cases. Based on such recognition, an approach named GASEN is proposed. GASEN trains a number of individual neural networks at first. Then it assigns random weights to the individual networks and employs a genetic algorithm to evolve those weights so that they can characterize to some extent the importance of the individual networks in constituting an ensemble. Finally it selects an optimum subset of individual networks based on the evolved weights to make up the ensemble. Experimental results show that, comparing with a popular ensemble approach, i.e. averaging all, and a theoretically optimum selective ensemble approach, i.e. enumerating, GASEN has preferable performance in generating ensembles with strong generalization ability in relatively small computational cost. This paper also analyzes the working mechanism of GASEN from the view of error-ambiguity decomposition, which reveals that GASEN improves generalization ability mainly through reducing the average generalization error of the individual neural networks constituting the ensemble.
Citations
|
4828
|
Genetic Algorithms
– Goldberg
- 1989
|
|
2138
|
UCI Repository of Machine Learning Databases
– Merz, Murphy
- 1996
|
|
1133
|
A decision-theoretic generalization of on-line learning and an application to boosting
– Freund, Schapire
- 1997
|
|
1034
|
An Introduction to the Bootstrap
– Efron, Tibshirani
- 1993
|
|
453
|
The strength of weak learnability
– Schapire
- 1990
|
|
368
|
Neural network ensembles
– Hansen, Salamon
- 1990
|
|
308
|
Neural network ensembles, cross validation, and active learning
– Krogh, Vedelsby
- 1995
|
|
243
|
When networks disagree: Ensemble methods for neural network Neural networks for speech and image processing
– Perrone, Cooper
- 1993
|
|
157
|
Learning representations by back-propagating errors. Nature
– Rumelhart, Hinton, et al.
- 1986
|
|
98
|
Estimating optimal transformations for multiple regression and correlation
– Breiman, Friedman
- 1985
|
|
82
|
Generating accurate and diverse members of a neural-network ensemble
– Opitz, Shavlik
- 1996
|
|
55
|
expert-level performance on a scienti c image analysis task by a system using combined arti cial neural networks
– Cherkauer, Human
- 1996
|
|
51
|
A Novel Objective Function for Improved Phoneme Recognition using Time-Delay Neural Networks
– Hampshire, Waibel
- 1990
|
|
51
|
Making use of population information in evolutionary artificial neural networks
– Yao, Liu
- 1998
|
|
49
|
Actively searching for an effective neural network ensemble
– Opitz, Shavlik
- 1996
|
|
42
|
A Genetic Algorithm for function OpTimization: A Matlab implementation
– Houck, Joines, et al.
- 1995
|
|
37
|
Stacked generalization, Neural Networks 5
– Wolpert
- 1992
|
|
32
|
Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks
– Maclin, Shavlik
- 1995
|
|
24
|
Pose invariant face recognition
– Huang, Chen, et al.
- 2000
|
|
24
|
Bagging predictors, Machine Learning 24
– Breiman
- 1996
|
|
20
|
X.: Ensemble learning via negative correlation. Neural Networks 12(10
– Liu, Yao
- 1999
|
|
20
|
Dynamically weighted ensemble neural networks for classification
– Jimenez, Walsh
- 1998
|
|
19
|
Optimal linear combination of neural networks for improving classification performance
– Ueda
- 2000
|
|
16
|
A case study on bagging, boosting and basic ensembles of neural networks for OCR
– Mao
- 1998
|
|
16
|
Classification of Seismic Signals by Integrating Ensembles of Neural Networks
– Shimshoni, Intrator
- 1998
|
|
15
|
Ensemble methods for handwritten digit recognition
– Hansen, Liisberg, et al.
- 1992
|
|
13
|
Adapting an Ensemble Approach for the Diagnosis of Breast Cancer
– Sharkey, Sharkey, et al.
- 1988
|
|
13
|
Boosting methodology for regression problems
– Ridgeway, Madigan, et al.
- 1999
|
|
13
|
Multidimensional additive spline approximation
– Friedman, Grosse, et al.
- 1983
|
|
12
|
Face Recognition Using Hybrid Classifier Systems
– Gutta, Wechsler
- 1996
|
|
4
|
Boosting a weak algorithm by majority
– Freund
- 1995
|
