D. MacKay. Gaussian Processes - A Replacement for Supervised Neural Networks? In Proc. Neural Inf. Proc. Syst., 1997. Lecture note.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....sets, e.g. the MNIST database is realistic. 1 Introduction Tipping s relevance vector machine (RVM) both achieves a sparse solution like the support vector machine (SVM) 2, 3] and the probabilistic predictions of Bayesian kernel machines based upon a Gaussian process (GP) priors over functions [4, 5, 6, 7, 8]. Sparsity is interesting both with respect to fast training and predictions and ease of interpretation of the solution. Probabilistic predictions are desirable because inference is most naturally formulated in terms of probability theory, i.e. we can manipulate probabilities through Bayes ....

David J. C. MacKay, "Gaussian processes - a replacement for supervised neural networks?," Lecture notes for a tutorial in Advances in Neural Information Processing Systems, 1997.

....objective function. In the next iteration, the optimum is searched on the improved model. This iterative procedure has been discussed by Torczon et al. 6] and similar procedures can be found in [7, 8, 9] In this paper, we analyze the second approach with a focus on Gaussian Process (GP) models [10]. GPs exhibit the key advantage of delivering an uncertainty measure in form of a standard deviation # for the predicted function value t. We address implementation issues like local modeling of the data, parallelization of the function evaluation and avoiding premature convergence. Finally, the ....

....and avoiding premature convergence. Finally, the proposed algorithm is compared to CMA on two test functions in order to illustrate the convergence properties of the algorithm. 2 Optimization Using Models 2. 1 Gaussian Process Models We define an empirical model using the notation of MacKay [10]: Let f(x) be an unknown function and R a data point in an L dimensional design space. From f , N samples are generated with XN = x 1 , x 2 , xN and the corresponding function values are t N = t 1 , t 2 , t N . The modeling task is to predict the function value t ....

[Article contains additional citation context not shown here]

MacKay, D.J.C.: Gaussian processes - a replacement for supervised neural networks? In: Lecture notes for a tutorial at NIPS. (1997)

....function: this would not generate positive semi definite covariance matrices. Instead we have to go back to square one. We can generate covariances functions (which are positive semidefinite) by convolving any symmetric kernel with itself, and so we can choose some more suitable kernels. Mackay [5] mentions the use of a top hat kernel (K(x; r) 1 if jx Gamma rj 1, zero elsewhere) to generate the covariance C(x; y) ae 1 Gamma jx Gamma yjfor jx Gamma yj 1 0 elsewhere Clearly this covariance function is of the type we are interested in. It is zero outside the region jx Gamma ....

D. J. C. Mackay. Gaussian processes - a replacement for supervised neural networks? In NIPS97 Tutorial, 1997.

....by convolving a Gaussian kernel with itself: C(x; y) Z exp( Gamma (x Gamma r) 2 2oe ) exp( Gamma (y Gamma r) 2 2oe ) We can generate covariances functions (which are positive semidefinite) by convolving any kernel with itself, and so we can choose some more suitable kernels. Mackay [42] mentions the use of a top hat kernel (K(x; r) 1 if jx Gamma rj 1, zero elsewhere) to generate the covariance C(x; y) 1 Gamma jx Gamma yjfor jx Gamma yj 1 0elsewhere Clearly this covariance function is of the type we are interested in. It is zero outside the region jx Gamma yj ....

D. J. C. Mackay. Gaussian processes - a replacement for supervised neural networks? In NIPS97 Tutorial, 1997.

....and general components. Only an brief explanation of Gaussian process models will be given here; for a detailed description of how Gaussian process models can be applied to regression and classification tasks, the reader should consult Williams and Rasmussen (1996) Rasmussen (1996) Neal (1997) Mackay (1997), or Gibbs and Mackay (1997) 3.1 Overview of Gaussian process models The Gaussian process models used in this paper can be seen as a type of nearest neighbor model that adaptively choose which input dimensions (or directions) are important when determining distance. Consider using a Gaussian ....

.... an brief explanation of Gaussian process models will be given here; for a detailed description of how Gaussian process models can be applied to regression and classification tasks, the reader should consult Williams and Rasmussen (1996) Rasmussen (1996) Neal (1997) Mackay (1997) or Gibbs and Mackay (1997). 3.1 Overview of Gaussian process models The Gaussian process models used in this paper can be seen as a type of nearest neighbor model that adaptively choose which input dimensions (or directions) are important when determining distance. Consider using a Gaussian process model for a regression ....

Mackay, D. J. (1997). Gaussian processes - a replacement for supervised neural networks? Lecture notes for a tutorial at NIPS 1997. Available at http://wol.ra.phy.cam.ac.uk/mackay.

No context found.

D. MacKay. Gaussian Processes - A Replacement for Supervised Neural Networks? In Proc. Neural Inf. Proc. Syst., 1997. Lecture note.

No context found.

D.J. MacKay. Gaussian Processes: A Replacement for Supervised Neural Networks? In Lecture Notes of Tutorial at Neural Information Processing Systems (NIPS'97), 1997.

No context found.

D. J. C. Mackay, "Gaussian Processes: A replacement for supervised Neural Networks?," Tech. Rep., Cavendish Laboratory, Cambridge University,

No context found.

MacKay, D.: Gaussian Processes - A Replacement for Supervised Neural Networks?. Lecture Notes for a Tutorial at NIPS (1997)

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC