Abstract

ii Non-parametric models and techniques enjoy a growing popularity in the field of machine learning, and among these Bayesian inference for Gaussian process (GP) models has recently received significant attention. We feel that GP priors should be part of the standard toolbox for constructing models relevant to machine learning in the same way as parametric linear models are, and the results in this thesis help to remove some obstacles on the way towards this goal. In the first main chapter, we provide a distribution-free finite sample bound on the difference between generalisation and empirical (training) error for GP classification methods. While the general theorem (the PAC-Bayesian bound) is not new, we give a much simplified and somewhat generalised derivation and point out the underlying core technique (convex duality) explicitly. Furthermore, the application to GP models is novel (to our knowledge). A central feature of this bound is that its quality depends crucially on task knowledge being encoded

Citations

374	Generalized belief propagation - Broadway, Yedidia, et al. - 2000
261	Using the Nyström method to speed up kernel machines - WILLIAMS, SEEGER
183	Prediction with Gaussian processes: From linear regression to linear prediction and beyond - Williams - 1998
135	On the Convergence Properties of the EM Algorithm for Gaussian Mixtures - Xu, Jordan - 1983
123	Bayesian classification with Gaussian processes - Williams, Barber - 1998
99	Correlation Theory of Stationary and Related Random Functions. Volume II: Supplementary Notes and References - Yaglom - 1987
23	Modified Cholesky factorizations in interior-point algorithms for linear programming - Wright - 1999
17	Gaussian regression and optimal finite dimensional linear models,” Neural networks and machine learning - Zhu, Williams, et al. - 1998