Manfred Opper and Ole Winther. GP classification and SVM: Mean field results and leave-one-out estimator. In Advances in Large Margin Classifiers. MIT Press, 1999.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... (see [54] 52] for a discussion of GP regression) A set of different techniques have been proposed to approximate the predictive distribution or moments thereof, based on Laplace approximations [55] Markov chain Monte Carlo [32] variational techniques [15] 42] or mean field approximations [34]. We will describe the Laplace approximation in detail, since the correspondence between GPC and SVC we would like to point out holds only if full Bayesian GPC is approximated in this way. In the rest of the paper we will use the term Gaussian process classification for this special approximation ....

....really is. The y vectors can be quite different, being implicitely defined by the systems y = KT ff GP ( y) and y = KT ff SV M ( y) respectively, these systems encode, via the definition of the dual variables, the completely different expectations we have in the predictions. Opper and Winther [34] show how to derive the SVC noise model starting from the GPC model. Their insightful discussion builds on the fact that common GPC noise models can be represented as the marginal expectation of a step function whose centre is a latent zero mean random slack variable. Trying to solve for the ....

Manfred Opper and Ole Winther. GP classification and SVM: Mean field results and leave-one-out estimator. In Advances in Large Margin Classifiers. MIT Press, 1999.

....mentioned in Section 2, SVR requires one more regularization parameter to trade off ffl against model complexity and training accuracy. is attractive in being logically consistent, simple and flexible. Recently, various Bayesian techniques have been applied to support vector classification (SVC) [2, 4, 7, 10, 13]. Here, we follow [2, 4] in applying the evidence framework [5] to SVR. The evidence framework is divided into three levels of inference, and is computationally equivalent to the type II maximum likelihood method in Bayesian statistics. Its use in feedforward neural networks [6] has allowed the ....

M. Opper and O. Winther. GP classification and SVM: Mean field results and leave-one-out estimator. In A.J. Smola, editor, Advances in Large Margin Classifiers. MIT, 1999.

....is the logit log(P ( 1jx) P ( Gamma1jx) of the target distribution. For this noise model the integral in (1) is not analytically tractable, but a range of approximative techniques based on Laplace approximations [16] Markov chain Monte Carlo [7] variational methods [2] or mean field algorithms [8] are known. We follow [16] The Laplace approach to GPC is to approximate the posterior P (y jD; by the Gaussian distribution N( y ; H Gamma1 ) where y = argmaxP (y jD; is the posterior mode and H = r 0 y ry ( Gamma log P (y jD; evaluated at y . Then it is easy to show that the ....

....so we cannot reduce SVC to a special case of a Gaussian process classification model. Although a generative model for SVC is given in [11] it is easier and less problematic to regard SVC as efficient approximation to a proper Gaussian process model. Various such models have been proposed (see [8], 4] In this work, we simply normalize the SVC loss pointwise, i.e. use a Gaussian process model with the normalized SVC loss g(t; y) 1 Gamma ty] log Z(y) Z(y) exp( Gamma[1 Gamma y] exp( Gamma[1 y] Note that g(t; y) is a close approximation of the (unnormalized) SVC ....

[Article contains additional citation context not shown here]

Manfred Opper and Ole Winther. GP classification and SVM: Mean field results and leave-one-out estimator. In Advances in Large Margin Classifiers. MIT Press, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC