Some Greedy Learning Algorithms for Sparse Regression and Classification with Mercer Kernels

DMCA

Some Greedy Learning Algorithms for Sparse Regression and Classification with Mercer Kernels (2002)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Prasanth B. Nair , Arindam Choudhury , Andy J. Keane , Carla Brodley , Andrea Danyluk

Venue:	Journal of Machine Learning Research
Citations:	23 - 1 self

BibTeX

@ARTICLE{Nair02somegreedy,
    author = {Prasanth B. Nair and Arindam Choudhury and Andy J. Keane and Carla Brodley and Andrea Danyluk},
    title = {Some Greedy Learning Algorithms for Sparse Regression and Classification with Mercer Kernels},
    journal = {Journal of Machine Learning Research},
    year = {2002},
    volume = {3},
    pages = {2002}
}

OpenURL

Abstract

We present some greedy learning algorithms for building sparse nonlinear regression and classification models from observational data using Mercer kernels. Our objective is to develop efficient numerical schemes for reducing the training and runtime complexities of kernel-based algorithms applied to large datasets. In the spirit of Natarajan's greedy algorithm (Natarajan, 1995), we iteratively minimize the L 2 loss function subject to a specified constraint on the degree of sparsity required of the final model until a specified stopping criterion is reached. We discuss various greedy criteria for basis selection and numerical schemes for improving the robustness and computational efficiency. Subsequently, algorithms based on residual minimization and thin QR factorization are presented for constructing sparse regression and classification models. During the course of the incremental model construction, the algorithms are terminated using model selection principles such as the minimum descriptive length (MDL) and Akaike's information criterion (AIC). Finally, experimental results on benchmark data are presented to demonstrate the competitiveness of the algorithms developed in this paper.

Keyphrases

mercer kernel sparse regression greedy learning algorithm classification model experimental result benchmark data runtime complexity basis selection incremental model construction greedy algorithm minimum descriptive length information criterion model selection principle observational data large datasets numerical scheme various greedy criterion computational efficiency kernel-based algorithm final model loss function subject thin qr factorization specified stopping criterion efficient numerical scheme sparse nonlinear regression residual minimization

DMCA