Covariance matrices are important in many areas of neural modelling. In Hopfield networks they are used to form the weight matrix which controls the autoassociative properties of the network. In Gaussian processes, which have been shown to be the infinite neuron limit of many regularised feedforward neural networks, covariance matrices control the form of Bayesian prior distribution over function space. This thesis examines interesting modifications to the standard covariance matrix methods to increase functionality or efficiency of these neural techniques. Firstly the problem of adapting Gaussian process priors to perform regression on switching regimes is tackled. This involves the use of block covariance matrices and Gibbs sampling methods. Then the use of Toeplitz methods is proposed for Gaussian process regression where sampling positions can be chosen. A comparison is made between Hopfield weight matrices, and sample covariances. This allows work on sample covariances to be used to estimate the eigenvalue structure of
|
1390
|
Introduction to the Theory of Neural Computation
– Hertz, Krogh, et al.
- 1991
|
|
1143
|
Matrix Computations
– Golub, Loan
- 1989
|
|
1007
|
Neural networks and physical systems with emergent collective computational abilities
– Hopfield
- 1982
|
|
800
|
Multilayer feedforward networks are universal approximators
– Hornik, Stinchcombe, et al.
- 1989
|
|
727
|
Spline Models for Observational Data
– Wahba
- 1990
|
|
445
|
Analog VLSI and Neural Systems
– Mead
- 1989
|
|
431
|
Learning representation by back propagating errors
– Rumelhart, Hinton, et al.
- 1986
|
|
410
|
A New View of the EM Algorithm that Justifies Incremental and Other Variants“, Learning in Graphical Models
– Neal, Hinton
- 1993
|
|
403
|
and T.W.Tank, “Neural computation of decisions in optimization problems
– Hopfield
- 1985
|
|
311
|
A practical Bayesian framework for backprop networks
– MacKay
- 1992
|
|
152
|
Probability Theory
– Loève
- 1955
|
|
142
|
Probability and Random Processes
– Grimmett, Stirzaker
- 1982
|
|
117
|
Modeling Brain Function
– Amit
- 1989
|
|
91
|
Evaluation of Gaussian Processes and Other Methods for Non-linear Regression
– Rasmussen
- 1996
|
|
74
|
carlo implementation of gaussian process models for bayesian regression and classification
– Neal
- 1997
|
|
69
|
Bayesian Classification with Gaussian Processes
– Williams, Barber
- 1998
|
|
69
|
The capacity of the Hopfield associative memory
– McEliece, Posner, et al.
- 1987
|
|
66
|
Improper Priors, Spline Smoothing and the Problem of Guarding Against Model Errors in Regression
– Wahba
- 1978
|
|
63
|
Learning algorithms with optimal stability in neural networks
– Krauth, Mézard
- 1987
|
|
58
|
Distribution of eigenvalues for some sets of random matrices
– Marcenko, Pastur
- 1967
|
|
57
|
Statistical mechanics of neural networks near saturation
– Amit, Gutfreund, et al.
- 1987
|
|
47
|
Efficient implementation of Gaussian processes. unpublished
– MacKay, Gibbs
- 1997
|
|
45
|
Learning in neural networks with material synapses. Neural Comput 6:957–982
– DJ, Fusi
- 1994
|
|
43
|
An algorithm for the inversion of finite Toeplitz matrices
– Trench
- 1964
|
|
41
|
Bayesian regularization and pruning using a Laplace prior
– Williams
- 1995
|
|
32
|
Gaussian processes: A replacement for supervised neural networks
– MacKay
- 1997
|
|
32
|
Collective computational properties of neural networks: New learning algorithms
– Personnaz, I, et al.
- 1986
|
|
27
|
Associative Recall of Memory Without Errors
– Kanter, Sompolinsky
- 1987
|
|
26
|
Statistical neurodynamics of associative memory
– Amari, Maginu
- 1988
|
|
24
|
Gaussian processes for bayesian classification via hybrid monte carlo
– Barber, Williams
- 1997
|
|
23
|
A memory which forgets
– Parisi
- 1986
|
|
22
|
The strong limits of random matrix spectra for sample matrices of independent elements
– Wachter
- 1978
|
|
21
|
Network of formal neurons and memory palimpsests Europhys Lett 1:535–542
– JP, Toulouse, et al.
- 1986
|
|
19
|
Circulant preconditioned Toeplitz least squares iterations
– Chan, Nagy, et al.
- 1994
|
|
18
|
Priors for infinite networks
– Neal
- 1994
|
|
18
|
On the Use of Evidence in Neural Networks
– Wolpert
- 1993
|
|
17
|
Bayesian numerical analysis
– Skilling
- 1993
|
|
17
|
The existence of persistent states
– Little
- 1974
|
|
16
|
Variational Gaussian process classifiers
– Gibbs, Mackay
- 2000
|
|
15
|
Regression with input-dependent noise: A Gaussian process treatment
– Goldberg, Williams, et al.
- 1998
|
|
15
|
A logical calculus of the ideas immanent in neural nets
– McCulloch, Pitts
- 1943
|
|
15
|
Spectral analysis of networks with random topologies
– Grenander, Silverstein
- 1977
|
|
14
|
Prediction with Gaussian processes
– Williams
- 1999
|
|
13
|
Iterative Retrieval of Sparsely Coded Associative Memory Patterns. Neural Networks
– Schwenker, Sommer, et al.
- 1996
|
|
12
|
Solvable models of working memories
– Mézard, JP, et al.
- 1986
|
|
11
|
Hyperparameters: optimise or integrate out
– Mackay
- 1994
|
|
11
|
Matrix Methods for Engineers and Scientists
– Barnett
- 1979
|
|
10
|
On curve fitting and optimal design for regression
– O'Hagan
- 1978
|
|
10
|
Information Storage and Retrieval in Spin-Glass-like neural networks
– Personnaz, I, et al.
- 1985
|
|
10
|
Multifractality in forgetful memories
– Behn, Hemmen, et al.
- 1993
|
