Download:
|
by Jian Zhang, Rong Jin, Yiming Yang, Alex G. Hauptmann
In Proceedings of the 20th International Conference on Machine Learning
http://www.cs.cmu.edu/~yiming/papers.yy/jzhang-icml03.ps.gz
Add To MetaCart
Abstract:
Logistic Regression (LR) has been widely used in statistics for many years, and has received extensive study in machine learning community recently due to its close relations to Support Vector Machines (SVM) and AdaBoost. In this paper, we use a modified version of LR to approximate the optimization of SVM by a sequence of unconstrained optimization problems. We prove that our approximation will converge to SVM, and propose an iterative algorithm called "MLRCG " which uses Conjugate Gradient as its inner loop. Multiclass version "MMLR-CG " is also obtained after simple modifications. We compare the MLR-CG with SVM light over di#erent text categorization collections, and show that our algorithm is much more efficient than SVM light when the number of training examples is very large. Results of the multiclass version MMLR-CG is also reported.
Citations
|
4514
|
Statistical Learning Theory
– Vapnik
- 1998
|
|
1410
|
Convex Analysis
– Rockafellar
- 1970
|
|
1004
|
Experiments with a new boosting algorithm
– Schapire
- 1996
|
|
961
|
Text Categorization with Support Vector Machines
– Joachims
- 1997
|
|
697
|
Making large-Scale SVM Learning Practical
– Joachims
- 1999
|
|
686
|
and Nonlinear Programming
– Luenberger, Linear
- 1984
|
|
543
|
Additive logistic regression: a statistical view of boosting
– Friedman, Hastie, et al.
|
|
541
|
Numerical Optimization
– Nocedal, Wright
|
|
495
|
Training of Support Vector Machine using Sequential Minimal Optimization
– Platt
- 1999
|
|
416
|
A re-examination of text categorization methods
– Yang, Liu
- 1999
|
|
101
|
Rcv1: A new benchmark collection for text categorization research
– Lewis, Yang, et al.
- 2004
|
|
56
|
Support vector machines for multi-class pattern recognition
– Weston, Watkins
- 1999
|
|
55
|
A study of approaches to hypertext categorization
– Yang, Slattery, et al.
|
|
43
|
Boosting and maximum likelihood for exponential models
– Lebanon, Lafferty
- 2001
|
|
35
|
Algorithms for maximum-likelihood logistic regression. http://lib.stat.cmu.edu/ minka/papers/logreg.html
– Minka
- 2000
|
|
19
|
The Nature of Statistical Learning Theory. 2nd Edition
– Vapnik
- 2001
|
|
5
|
Stable Exponential-Penalty Algorithm with Superlinear Convergence
– Cominetti, Dussault
- 1994
|
|
1
|
Uniqueness of the SVM Solution. NIPS
– Burges, Crisp
- 1999
|
