Benchmarking Least Squares Support Vector Machine Classifiers

DMCA

Benchmarking Least Squares Support Vector Machine Classifiers (2001)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Tony Van Gestel , Johan A. K. Suykens , Bart Baesens , Stijn Viaene , Jan Vanthienen , Guido Dedene , Bart De Moor , Joos Vandewalle

Venue:	NEURAL PROCESSING LETTERS
Citations:	476 - 46 self

BibTeX

@INPROCEEDINGS{Gestel01benchmarkingleast,
    author = {Tony Van Gestel and Johan A. K. Suykens and Bart Baesens and Stijn Viaene and Jan Vanthienen and Guido Dedene and Bart De Moor and Joos Vandewalle},
    title = {Benchmarking Least Squares Support Vector Machine Classifiers},
    booktitle = {NEURAL PROCESSING LETTERS},
    year = {2001},
    pages = {293--300},
    publisher = {}
}

OpenURL

Abstract

In Support Vector Machines (SVMs), the solution of the classification problem is characterized by a (convex) quadratic programming (QP) problem. In a modified version of SVMs, called Least Squares SVM classifiers (LS-SVMs), a least squares cost function is proposed so as to obtain a linear set of equations in the dual space. While the SVM classifier has a large margin interpretation, the LS-SVM formulation is related in this paper to a ridge regression approach for classification with binary targets and to Fisher's linear discriminant analysis in the feature space. Multiclass categorization problems are represented by a set of binary classifiers using different output coding schemes. While regularization is used to control the effective number of parameters of the LS-SVM classifier, the sparseness property of SVMs is lost due to the choice of the 2-norm. Sparseness can be imposed in a second stage by gradually pruning the support value spectrum and optimizing the hyperparameters during the sparse approximation procedure. In this paper, twenty public domain benchmark datasets are used to evaluate the test set performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances. These SVM and LS-SVM performances are consistently very good when compared to a variety of methods described in the literature including decision tree based algorithms, statistical algorithms and instance based learning methods. We show on ten UCI datasets that the LS-SVM sparse approximation procedure can be successfully applied.

Keyphrases

square support vector machine classifier ls-svm classifier feature space modified version hyperparameter selection linear set ls-svm sparse approximation procedure ls-svm performance binary target large margin interpretation dual space support value spectrum ridge regression approach discriminant analysis classification problem comparable test set performance different output rbf kernel multiclass categorization problem least square svm classifier sparse approximation procedure decision tree sparseness property statistical algorithm square cost function effective number support vector machine twenty public domain benchmark datasets second stage quadratic programming ten uci datasets standard cross-validation procedure ls-svm formulation binary classifier svm classifier radial basis function learning method

DMCA