Results 1  10
of
26
Least Squares Linear Discriminant Analysis
"... Linear Discriminant Analysis (LDA) is a wellknown method for dimensionality reduction and classification. LDA in the binaryclass case has been shown to be equivalent to linear regression with the class label as the output. This implies that LDA for binaryclass classifications can be formulated as ..."
Abstract

Cited by 51 (6 self)
 Add to MetaCart
(Show Context)
Linear Discriminant Analysis (LDA) is a wellknown method for dimensionality reduction and classification. LDA in the binaryclass case has been shown to be equivalent to linear regression with the class label as the output. This implies that LDA for binaryclass classifications can be formulated as a least squares problem. Previous studies have shown certain relationship between multivariate linear regression and LDA for the multiclass case. Many of these studies show that multivariate linear regression with a specific class indicator matrix as the output can be applied as a preprocessing step for LDA. However, directly casting LDA as a least squares problem is challenging for the multiclass case. In this paper, a novel formulation for multivariate linear regression is proposed. The equivalence relationship between the proposed least squares formulation and LDA for multiclass classifications is rigorously established under a mild condition, which is shown empirically to hold in many applications involving highdimensional data. Several LDA extensions based on the equivalence relationship are discussed. 1.
Combining Discriminant Models with new MultiClass SVMs
, 2000
"... The idea of combining models instead of simply selecting the best one, in order to improve performance, is well known in statistics and has a long theoretical background. However, making full use of theoretical results is ordinarily subject to the satisfaction of strong hypotheses (weak correlati ..."
Abstract

Cited by 48 (10 self)
 Add to MetaCart
The idea of combining models instead of simply selecting the best one, in order to improve performance, is well known in statistics and has a long theoretical background. However, making full use of theoretical results is ordinarily subject to the satisfaction of strong hypotheses (weak correlation among the errors, availability of large training sets, possibility to rerun the training procedure an arbitrary number of times, etc.). In contrast, the practitioner who has to make a decision is frequently faced with the dicult problem of combining a given set of pretrained classiers, with highly correlated errors, using only a small training sample. Overtting is then the main risk, which cannot be overcome but with a strict complexity control of the combiner selected. This suggests that SVMs, which implement the SRM inductive principle, should be well suited for these dicult situations. Investigating this idea, we introduce a new family of multiclass SVMs and assess them as ensemble methods on a realworld problem. This task, protein secondary structure prediction, is an open problem in biocomputing for which model combination appears to be an issue of central importance. Experimental evidence highlights the gain in quality resulting from combining some of the most widely used prediction methods with our SVMs rather than with the ensemble methods traditionally used in the eld. The gain is increased when the outputs of the combiners are postprocessed with a simple DP algorithm.
Learning to discriminate between ligandbound and disulfidebound cysteines
 Protein Engineering Design and Selection
, 2004
"... disulfide bound cysteines ..."
(Show Context)
On kernelization of supervised mahalanobis distance learners
 Computing Research Repoisitory (CoRR
"... Abstract. This paper contains three contributions to the problem of learning a Mahalanobis distance. First, a general framework for kernelizing Mahalanobis distance learners is presented. The framework allows existing algorithms to learn a Mahalanobis distance in a feature space associated with a pr ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
(Show Context)
Abstract. This paper contains three contributions to the problem of learning a Mahalanobis distance. First, a general framework for kernelizing Mahalanobis distance learners is presented. The framework allows existing algorithms to learn a Mahalanobis distance in a feature space associated with a prespecified kernel function. The framework is then used for kernelizing three wellknown learners, namely, “neighborhood component analysis”, “large margin nearest neighbors ” and “discriminant neighborhood embedding”; open problems of recent works are thus solved. Second, while the truths of representer theorems are just assumptions in previous papers related to ours, here representer theorems in the context of kernelized Mahalanobis distance learners are formally proven. Third, unlike previous works which demand cross validation to select a kernel, an inductive kernel alignment method based on quadratic programming is derived in this paper and is used to automatically select an efficient kernel function. Numerical results on various realworld datasets are presented.
Model selection for multiclass SVMs
 In ASMDA’05
, 2005
"... Abstract. In the framework of statistical learning, fitting a model to a given problem is usually done in two steps. First, model selection is performed, to set the values of the hyperparameters. Second, training results in the selection, for this set of values, of a function performing satisfactori ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
Abstract. In the framework of statistical learning, fitting a model to a given problem is usually done in two steps. First, model selection is performed, to set the values of the hyperparameters. Second, training results in the selection, for this set of values, of a function performing satisfactorily on the problem. Choosing the values of the hyperparameters remains a difficult task, which has only been addressed so far in the case of biclass SVMs. We derive here a solution dedicated to MSVMs. It is based on a new bound on the risk of large margin classifiers. Keywords: Multiclass SVMs, hyperparameters, soft margin parameter. 1
Discriminant Analysis for Dimensionality Reduction: An Overview of Recent Developments
"... Many biometric applications such as face recognition involve data with a large number of features [1–3]. Analysis of such data is challenging due to the curseofdimensionality [4, 5], which states that an enormous number of samples are required to perform accurate predictions on problems with a high ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Many biometric applications such as face recognition involve data with a large number of features [1–3]. Analysis of such data is challenging due to the curseofdimensionality [4, 5], which states that an enormous number of samples are required to perform accurate predictions on problems with a high dimensionality. Dimensionality
Estimating the class posterior probabilities in protein secondary structure prediction
 in PRIB’11
, 2011
"... Abstract. Support vector machines, let them be biclass or multiclass, have proved efficient for protein secondary structure prediction. They can be used either as sequencetostructure classifier, structuretostructure classifier, or both. Compared to the classifier most commonly found in the mai ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Support vector machines, let them be biclass or multiclass, have proved efficient for protein secondary structure prediction. They can be used either as sequencetostructure classifier, structuretostructure classifier, or both. Compared to the classifier most commonly found in the main prediction methods, the multilayer perceptron, they exhibit one single drawback: their outputs are not class posterior probability estimates. This paper addresses the problem of postprocessing the outputs of multiclass support vector machines used as sequencetostructure classifiers with a structuretostructure classifier estimating the class posterior probabilities. The aim of this comparative study is to obtain improved performance with respect to both criteria: prediction accuracy and quality of the estimates.
Ensemble Methods of Appropriate Capacity for MultiClass Support Vector Machines
"... Abstract. Roughly speaking, there is one single model of pattern recognition support vector machine (SVM), with variants of lower popularity. On the contrary, among the different multiclass SVMs (MSVMs) published, none is clearly favoured. Although several comparative studies between MSVMs and de ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Roughly speaking, there is one single model of pattern recognition support vector machine (SVM), with variants of lower popularity. On the contrary, among the different multiclass SVMs (MSVMs) published, none is clearly favoured. Although several comparative studies between MSVMs and decomposition methods have been reported, no attention had been paid so far to the combination of those models. We investigate the combination of MSVMs with low capacity linear ensemble methods that estimate the class posterior probabilities. Keywords: Ensemble methods, MSVMs, Capacity control. 1
On Kernelizing Mahalanobis Distance Learning Algorithms On Kernelizing Mahalanobis Distance Learning Algorithms
, 804
"... This paper focuses on the problem of kernelizing an existing supervised Mahalanobis distance learner. The following features are included in the paper. Firstly, three popular learners, namely, “neighborhood component analysis”, “large margin nearest neighbors” and “discriminant neighborhood embeddin ..."
Abstract
 Add to MetaCart
(Show Context)
This paper focuses on the problem of kernelizing an existing supervised Mahalanobis distance learner. The following features are included in the paper. Firstly, three popular learners, namely, “neighborhood component analysis”, “large margin nearest neighbors” and “discriminant neighborhood embedding”, which do not have kernel versions are kernelized in order to improve their classification performances. Secondly, an alternative kernelization framework called “KPCA trick ” is presented. Implementing a learner in the new framework gains several advantages over the standard framework. Thirdly, while the truths of representer theorems are just assumptions in previous papers related to ours, here, representer theorems are formally proven. The proofs validate both the kernel trick and the KPCA trick in the context of Mahalanobis distance learning. Fourthly, unlike previous works which always apply brute force methods to select a kernel, we investigate two approaches which can be efficiently adopted to construct an appropriate kernel for a given dataset. Finally, numerical results on various realworld datasets are presented to show the performances of the kernelized algorithms.