Results 1 
5 of
5
Asymptotic normality of support vector machine variants and other regularized kernel methods
 Journal of Multivariate Analysis
"... ar ..."
Asymptotic Confidence Sets for General Nonparametric Regression and Classification by Regularized Kernel Methods
, 2012
"... Regularized kernel methods such as, e.g., support vector machines and leastsquares support vector regression constitute an important class of standard learning algorithms in machine learning. Theoretical investigations concerning asymptotic properties have manly focused on rates of convergence durin ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Regularized kernel methods such as, e.g., support vector machines and leastsquares support vector regression constitute an important class of standard learning algorithms in machine learning. Theoretical investigations concerning asymptotic properties have manly focused on rates of convergence during the last years but there are only very few and limited (asymptotic) results on statistical inference so far. As this is a serious limitation for their use in mathematical statistics, the goal is to fill this gap. Based on asymptotic normality of many of these methods [1], a strongly consistent estimator for the unknown covariance matrix of the limiting normal distribution is derived. In this way, we obtain asymptotically correct confidence sets for ψ(fP,λ0) where fP,λ0 denotes the minimizer of the regularized risk in the reproducing kernel Hilbert space H and ψ: H → Rm is any Hadamarddifferentiable functional. Applications include (multivariate) pointwise confidence sets for values
A Consistent Information Criterion for Support Vector Machines in Diverging Model Spaces
, 2016
"... Abstract Information criteria have been popularly used in model selection and proved to possess nice theoretical properties. For classification, ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract Information criteria have been popularly used in model selection and proved to possess nice theoretical properties. For classification,
From the Support Vector Machine to the Bounded Constraint Machine
"... The Support Vector Machine (SVM) has been successfully applied for classification problems in many different fields. It was originally proposed using the idea of searching for the maximum separation hyperplane. In this article, in contrast to the criterion of maximum separation, we explore alternati ..."
Abstract
 Add to MetaCart
(Show Context)
The Support Vector Machine (SVM) has been successfully applied for classification problems in many different fields. It was originally proposed using the idea of searching for the maximum separation hyperplane. In this article, in contrast to the criterion of maximum separation, we explore alternative searching criteria which result in the new method, the Bounded Constraint Machine (BCM). Properties and performance of the BCM are explored. To connect the BCM with the SVM, we investigate the Balancing Support Vector Machine (BSVM), which can be viewed as a bridge from the SVM to the BCM. The BCM is shown to be an extreme case of the BSVM. Theoretical properties such as Fisher consistency and asymptotic distributions for coefficients are derived, and the entire solution path of the BSVM is developed. Our numerical results demonstrate how the BSVM and the BCM work compared to the SVM.
PredictionBased Structured Variable Selection through the Receiver Operating Characteristic Curves
, 2011
"... Summary. In many clinical settings, a commonly encountered problem is to assess accuracy of a screening test for early detection of a disease. In these applications, predictive performance of the test is of interest. Variable selection may be useful in designing a medical test. An example is a resea ..."
Abstract
 Add to MetaCart
Summary. In many clinical settings, a commonly encountered problem is to assess accuracy of a screening test for early detection of a disease. In these applications, predictive performance of the test is of interest. Variable selection may be useful in designing a medical test. An example is a research study conducted to design a new screening test by selecting variables from an existing screener with a hierarchical structure among variables: there are several root questions followed by their stem questions. The stem questions will only be asked after a subject has answered the root question. It is therefore unreasonable to select a model that only contains stem variables but not its root variable. In this work, we propose methods to perform variable selection with structured variables when predictive accuracy of a diagnostic test is the main concern of the analysis. We take a linear combination of individual variables to form a combined test. We then maximize a direct summary measure of the predictive performance of the test, the area under a receiver operating characteristic curve (AUC of an ROC), subject to a penalty function to control for overfitting. Since maximizing empirical AUC of the ROC of a combined test is a complicated nonconvex problem (Pepe, Cai, and Longton, 2006, Biometrics 62, 221–229), we explore the connection between the empirical AUC and a support vector machine (SVM). We cast the problem of maximizing predictive performance of a combined test as a penalized SVM problem and apply a reparametrization to impose the hierarchical structure among variables. We also