DMCA
Novel distance-based SVM kernels for infinite ensemble learning (2005)
Cached
Download Links
| Venue: | In Proceedings of the 12th International Conference on Neural Information Processing |
| Citations: | 8 - 3 self |
BibTeX
@INPROCEEDINGS{Lin05noveldistance-based,
author = {Hsuan-tien Lin and Ling Li},
title = {Novel distance-based SVM kernels for infinite ensemble learning},
booktitle = {In Proceedings of the 12th International Conference on Neural Information Processing},
year = {2005},
pages = {761--766}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract — Ensemble learning algorithms such as boosting can achieve better performance by averaging over the predictions of base hypotheses. However, most existing algorithms are limited to combining only a finite number of hypotheses, and the generated ensemble is usually sparse. It has recently been shown that the support vector machine (SVM) with a carefully crafted kernel can be used to construct a nonsparse ensemble of infinitely many hypotheses. Such infinite ensembles may surpass finite and/or sparse ensembles in learning performance and robustness. In this paper, we derive two novel kernels, the stump kernel and the perceptron kernel, for infinite ensemble learning. The stump kernel embodies an infinite number of decision stumps, and measures the similarity between examples by the ℓ1-norm distance. The perceptron kernel embodies perceptrons, and works with the ℓ2-norm distance. Experimental results show that SVM with these kernels is superior to boosting with the same base hypothesis set. In addition, SVM with these kernels has similar performance to SVM with the Gaussian kernel, but enjoys the benefit of faster parameter selection. These properties make the kernels favorable choices in practice. I.
Keyphrases
infinite ensemble learning novel distance-based svm kernel base hypothesis stump kernel infinite number finite number many hypothesis perceptron kernel infinite ensemble abstract ensemble decision stump nonsparse ensemble kernel favorable choice similar performance novel kernel sparse ensemble 2-norm distance 1-norm distance support vector machine faster parameter selection perceptron kernel embodies perceptrons gaussian kernel experimental result
