Abstract

ii Non-parametric models and techniques enjoy a growing popularity in the field of machine learning, and among these Bayesian inference for Gaussian process (GP) models has recently received significant attention. We feel that GP priors should be part of the standard toolbox for constructing models relevant to machine learning in the same way as parametric linear models are, and the results in this thesis help to remove some obstacles on the way towards this goal. In the first main chapter, we provide a distribution-free finite sample bound on the difference between generalisation and empirical (training) error for GP classification methods. While the general theorem (the PAC-Bayesian bound) is not new, we give a much simplified and somewhat generalised derivation and point out the underlying core technique (convex duality) explicitly. Furthermore, the application to GP models is novel (to our knowledge). A central feature of this bound is that its quality depends crucially on task knowledge being encoded

Citations

308	Y: Generalized belief propagation - Yedidia, Freeman, et al.
207	Using the nyström method to speed up kernel machines - Williams, Seeger
160	Prediction with Gaussian processes: From linear regression to linear prediction and beyond - Williams - 1998
114	On convergence properties of the EM algorithm for Gaussian mixtures - Xu, Jordan - 1996
103	Bayesian classification with Gaussian processes - Williams, Barber - 1998
78	Correlation Theory of Stationary and Related Random Functions - Yaglom - 1987
22	Modified Cholesky factorizations in interior-point algorithms for linear programming - Wright - 1999
15	Gaussian regression and optimal finite dimensional linear models, in Neural networks and machine learning - Zhu, Williams, et al. - 1998