DMCA
The EM-EP Algorithm for Gaussian Process Classification Hyun-Chul Kim and Zoubin Ghahramani (2003)
| Citations: | 17 - 4 self |
BibTeX
@MISC{Kim03theem-ep,
author = {Hyun-chul Kim and Zoubin Ghahramani},
title = {The EM-EP Algorithm for Gaussian Process Classification Hyun-Chul Kim and Zoubin Ghahramani},
year = {2003}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Gaussian process classifiers (GPCs) are fully statistical kernel classification models derived from Gaussian processes for regression. In GPCs, the probability of belonging to a certain class at an input location is monotonically related to the value of some latent function at that location. Starting from a prior over this latent function, the data are used to infer both the posterior over the latent function and the values of hyperparameters determining various aspects of the function. GPCs can also be viewed as graphical models with latent variables. Based on the work of [1, 2], we present an approximate EM algorithm, the EM-EP algorithm for learning both the latent function and the hyperparameters of a GPC. The algorithm alternates the following steps until convergence. In the E-step, given the hyperparameters, a density for the latent variables defining the latent function is computed via the Expectation-Propagation (EP) algorithm [1, 2]. In the M-step, given the density for the latent values, the hyperparameters are selected to maximize a variational lower bound on the marginal likelihood (i.e. the model evidence). This algorithm is found to converge in practice and provides an efficient Bayesian framework for learning hyperparameters of the kernel. We examine the role of various different hyperparameters which model labeling errors, the lengthscales (i.e. relevances) of different features, and sharpness of the decision boundary. The added flexibility these provide results in significantly improved performance. Experimental results on synthetic and real data sets show that the EM-EP algorithm works well, with GPCs giving equal or better performance than support vector machines (SVMs) on all data sets tested. 1
Keyphrases
em-ep algorithm latent function gaussian process classification hyun-chul kim zoubin ghahramani gaussian process gaussian process classifier various aspect certain class model evidence efficient bayesian framework decision boundary real data set added flexibility statistical kernel classification model latent variable following step approximate em algorithm latent value input location marginal likelihood support vector machine graphical model data set various different hyperparameters experimental result different feature
