B. E. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classi ers. In Computational Learing Theory, pages 144-152, 1992.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....that avoid such explicit mappings by using kernel functions have become popular. The main idea is to construct algorithms that only a ord dot products of pattern vectors which can be computed eciently in high dimensional spaces. Examples of this type of algorithms are the Support Vector Machine [2] and Kernel Principal Component Analysis [9] In the following we will show that it is also possible to formulate classical discriminant analysis exclusively in terms of dot products which allows the use of kernel methods to construct a nonlinear variant of the algorithm. We call this technique ....

Boser B., Guyon I., Vapnik V. (1992) An training algorithm for optimal margin classi ers, Fifth Annual Workshop on Computational Learning Theory, Pittsburgh ACM, pp. 144-152

....obtained on realworld benchmark tasks are presented in section 5. This work was supported by the European Commission, grant number IST 99 11764, as part of its Framework V IST programme. The key results are summarised in section 6. 2 Support Vector Classi cation The support vector machine [1, 2], given labelled training data f(x i ; y i )g i=1 ; x i 2 X R d ; y i 2 f1; 1g; constructs a maximal margin linear classi er in a high dimensional feature space, x) de ned by a positive de nite kernel function, k(x; x ) specifying an inner product in the feature space, x) ....

B. Boser, I. Guyon, and V. N. Vapnik, \A training algorithm for optimal margin classi- ers," in Proceedings of the fth annual workshop on computational learning theory, (Pittsburgh), pp. 144-152, ACM, 1992.

....be huge, to an extent that it can fail to t in memory. Furthermore, its computation can be time consuming. A way to bypass this diculty consists in applying to the algorithm chosen a decomposition technique. This approach was already applied in the early works dealing with SVMs, works exposed in [2]. The chunking method used was the one described in [13] Addendum I, in the particular case of a linear model. The main decomposition techniques introduced afterwards are all based on the solution of the dual problem in the case when the value of some of (most of actually) the variables are xed. ....

B. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classi ers. In COLT'92, pages 144-152, 1992.

.... even the very best methods are typically quadratic) Therefore, many algorithms and implementation techniques have been developed for training SVMs eciently; see, e.g. SS98, BMS00] Among the proposed speed up techniques, the subset selection [SS98] has been used as an e ective heuristic [BGV92]. Roughly speaking, the subset selection is a technique for the speedup of SVM training by dividing the original QP problem into small pieces, thereby reducing the size of each QP problem. Well known variations of subset selection techniques are chunking, decomposition, and sequential minimal ....

B.E. Boser, I.M. Guyon, and V.N. Vapnik, A training algorithm for optimal margin classi ers, in Proc. 5th Ann. Conf. on Computational Learning Theory (COLT'92), ACM, 144 152, 1992.

....45 different models, one for each pair of classes. This scheme was already used to solve multiclass recognition problems with linear decision functions as in the Ho Kashyap classier. It is commonly referred as Pairwise strategy in contrast with the well known One Against Others strategy [3, 4]. We tried KMOD as well as RBF kernel and a polynomial kernel on NIST database using both of the learning strategies. We used a subset of 20,000 images from the hsf 123 part for training and 10,000 images from the hsf 7 part for testing. From each image we extract 272 features that well ....

B. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classiers. In Fifth Annual Workshop on Computational Learning Theory, Pittsburg, 1992.

....procedure for solving the SVC is usually based on Quadratic Programming (QP) which presents some inherent limitations, mainly the computational complexity and memory requirements for large training data sets. This problem is typically avoided by dividing the QP problem into sets of smaller ones [6, 1, 7, 11], that are iteratively solved in order to reach the SVC solution for the whole set of training samples. These schemes rely on an optimizing engine, QP, and in the sample selection strategy for each sub problem, in order to obtain a fast solution for the SVC. An Iterative Re Weighted Least Squares ....

....and G b3 = y S3 ( S3 G bin . 6. Go to step 1 and repeat until convergence. Table 1: IRWLS SVC algorithm. The IRWLS SVC procedure has to be slightly modi ed in order to be used inside a chunking scheme as the one proposed in [8, 6] such that it can be directly applied in the one proposed in [1]. A chunking scheme is needed to solve the SVC whenever H is too large to t into memory. In those cases, several SVC with a reduced set of training samples are iteratively solved until the solution for the whole set is found. The samples are divide into a working set, Sw , which is solved as a ....

B. E. Boser, I. M. Guyon, and V. Vapnik. A training algorithm for optimal margin classiers. In 5th Annual Workshop on Computational Learning Theory, Pittsburg, U.S.A., 1992.

....of new kernels recent years have witnessed the development of new methods, globally referred to as kernel methods, to perform various data mining algorithm from the knowledge of the kernel matrix only. Apart from the most famous support vector machine algorithm for classi cation and regression [BGV92, Vap98] other kernel methods include principal component analysis [SSM99] clustering [BHHSV01] Fisher discriminants [MRW 99] or independent component analysis [BJ01] These recent developments open the door to new analysis opportunities which we believe can be particularly suited to the ....

B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classi ers. In Proceedings of the 5th annual ACM workshop on Computational Learning Theory, pages 144-152. ACM Press, 1992.

....no additional e ort. Essentially we propose to perform a density estimation not in the original space but in the feature space of nonlinearly transformed data. The nonlinearity enters in terms of Mercer kernels [6] which have been extensively used in the classi cation and support vector community [1, 2], but which have apparently been studied far less in the eld of density estimation. In the present section we present the method of density estimation, discuss its relation to kernel principal component analysis (kernel PCA) 23] and to the Parzen estimator [20, 19] and propose estimates of the ....

B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classiers. In D. Haussler, editor, Proc. of the 5th Annual ACM Workshop on Comput. Learning Theory, pages 144-152, Pittsburgh, PA, 1992. ACM Press.

.... drawbacks, namely they assume some prior knowledge about the structure of the nonlinearity [8] or the number of underlying classes [3] or they involve an intricate model construction [2] An elegant and promising way to avoid these drawbacks is to employ feature spaces induced by Mercer kernels [1], in order to indirectly model a nonlinear transformation (x) of the original data from a space X into a potentially in nite dimensional space Y , aspiring a simpler distribution of the mapped data in Y . The search for an appropriate nonlinearity is replaced by the search for an appropriate ....

....For small values of one obtains the weighted sum of the single class energies (14) E(z) 1 P i c i X i c i E i (z) const for 1 : 15) The limit 1 gives their minimum: lim 1 E(z) min i E i (z) const. We compared our approach (12) for a Gaussian radial basis function kernel[1] k(x; y) exp jjx yjj 2 2 2 (16) to the supervised case (13) on an arti cial training set of 2D points, which were sampled from three di erent Gaussian distributions. The training data and the level lines of the respective energies are depicted in Figure 3. The comparison shows ....

B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classiers. In D. Haussler, editor, Proc. of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144-152, Pittsburgh, PA, 1992. ACM Press.

....ridge regression, neural networks October 30, 2001 DRAFT 3 I. Introduction Both theory and practice have pointed to the concept of the margin of a classi er as being central to the success of a new generation of learning algorithms. This is explicitly true of Support Vector Machines (SVMs) [9], 12] which in their simplest form implement maximal margin hyperplanes in a high dimensional feature space, but has also been shown to be the case for boosting algorithms such as Adaboost [24] Increasing the margin has been shown to implement a capacity control through datadependent structural ....

B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classiers. In D. Haussler, October 30,

....8:25; p.18 Linear Programming Boosting 19 Hard Margin Solution No Noise Case Figure 1. No noise Hard Margin LP solution for two con dence rated hypotheses. Left is the separation in label space. Right is the separation in dual or margin space. precursors to Boser et al. s Support Vector Machine (Boser et al. 1992; Cortes Vapnik, 1995) the soft margin RLP performed uniformly better than the hard margin MSM. In this section we will examine the critical di erence between hard and soft margin classi ers geometrically through a simple example. This discussion will also illustrate some of the practical ....

Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classiers. In Haussler, D. (Ed.), Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144-152. ACM Press.

No context found.

B. E. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classi ers. In Computational Learing Theory, pages 144-152, 1992.

No context found.

B. E. Boser, I. M. Guyon, and V. N. Vapnik, \A training algorithm for optimal margin classi ers," in Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, D. Haussler, Ed., Pittsburgh, PA, July 1992, pp. 144-152, ACM Press.

No context found.

B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classi ers. In David Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144-152, Pittsburgh, PA, July 1992. ACM Press.

No context found.

B. E. Boser, I. M. Guyon, V. N. Vapnik, \A Training Algorithm for Optimal Margin Classi ers", Proceedings of the Fifth Annual Workshop on Computational Learning Theory, 1992.

No context found.

B.E. Boser, I.M. Guyon, and V.N. Vapnik. A training algorithm for optimal margin classi ers. In D. Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144-152, Pittsburgh, 1992. ACM Press.

No context found.

B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classi ers. In D. Haussler, editor, ########### ## ### ### ###### ### ##### #### ## ############# ######## ######, pages 144{ 152, Pittsburgh, PA, July 1992. ACM Press.

No context found.

B. Boser, I. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classi ers. In D. Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144-152. ACM Press, 1992.

No context found.

B. E. Boser, I. M. Guyon, V. N. Vapnik, \A Training Algorithm for Optimal Margin Classi ers", Computational Learing Theory, 1992.

No context found.

B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classi ers. In D. Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144-152, Pittsburgh, PA, July 1992. ACM Press.

No context found.

B. Boser, I. Guyon, and V. N. Vapnik, \A training algorithm for optimal margin classi- ers," in ########### ## ### #### ###### ######## ## ############# ######## ######, (Pittsburgh), pp. 144-152, ACM, 1992.

No context found.

B. Boser, I. Guyon, and V. N. Vapnik, \A training algorithm for optimal margin classi- ers," in Proceedings of the fth annual workshop on computational learning theory, (Pittsburgh), pp. 144-152, ACM, 1992.

No context found.

B. Boser, I. Guyon, and V. N. Vapnik, \A training algorithm for optimal margin classi- ers," in Proceedings of the fth annual workshop on computational learning theory, (Pittsburgh), pp. 144-152, ACM, 1992.

No context found.

B. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classi ers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pages 144-152, 1992.

No context found.

Boser, B. E., Guyon, I. M. & Vapnik, V. (1992). A training algorithm for optimal margin classiers. In Fifth Annual Workshop on Computational Learning Theory, ACM, Pittsburgh.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC