Abstract:
Abstract. Hierarchical probabilistic synthesis and analysis models have recently been suggested as architectures for performing density estimation. Strict hierarchies makes it easy to evaluate generative or synthetic probabilities. However, both theoretical and neurobiological considerations weigh in favour of integrating lateral influences within a layer together with top-down and bottom up influences from lower and higher layers. This is known to be computationally tricky. We suggest a new recurrent sampling model and show that has the appropriate structure and behaviour for the analysis model for linear and Gaussian factor analysis. Then we extend this model to the case of binary stochastic units.
Citations
|
4539
|
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
– Pearl
- 1988
|
|
2405
|
Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images
– Geman, Geman
- 1984
|
|
442
|
Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381:607–609
– Olshausen, Field
- 1996
|
|
226
|
Learning and relearning in Boltzmann machines
– Hinton, Sejnowski
- 1986
|
|
185
|
The wake-sleep algorithm for unsupervised neural networks
– Hinton, Dayan, et al.
- 1995
|
|
158
|
The helmholtz machine
– Dayan, Hinton, et al.
- 1995
|
|
146
|
Natural gradient works efficiently in learning
– Amari
- 1998
|
|
140
|
Speed of processing in the human visual system
– Thorpe, Fize, et al.
- 1996
|
|
113
|
Generative models for discovering sparse distributed representations
– Hinton, Ghahramani
- 1997
|
|
105
|
Mean field theory for sigmoid belief networks
– Saul, Jaakkola, et al.
- 1996
|
|
99
|
An Introduction to Latent Variable Models
– Everitt
- 1984
|
|
94
|
Autoencoders, minimum description length, and helmholtz free energy
– Hinton, Zemel
- 1994
|
|
74
|
EM algorithms for ML factor analysis
– Rubin, Thayer
- 1982
|
|
57
|
A minimum description length framework for unsupervised learning
– Zemel
- 1993
|
|
55
|
Neuronal architectures for pattern-theoretic problems
– Mumford
- 1994
|
|
41
|
Variational Methods for Inference and Estimation in Graphical Models
– Jaakkola
- 1997
|
|
27
|
Does the wake-sleep algorithm produce good density estimators
– Frey, Hinton, et al.
- 1995
|
|
25
|
Fast learning by bounding likelihoods in sigmoid type belief networks
– Jaakkola, Saul, et al.
- 1996
|
|
21
|
Factor analysis using delta-rule wake-sleep learning
– Neal, Dayan
- 1996
|
|
20
|
Activity changes in early visual cortex reflect monkeysx percepts during binocular rivalry. Nature 379
– Leopold, Logothetis
- 1996
|
|
11
|
Dynamic model of visual memory predicts neural response properties in the visual cortex
– Rao, Ballard
- 1995
|
|
9
|
Development of forward and feedback connections between areas V1 and V2 of human visual cortex
– Burkhalter
- 1993
|
|
5
|
Stochastic cellular automata with Gibbsian invariant measures
– Marroquin
- 1991
|
|
2
|
A Mean Field Algorithm for Unsupervised Neural Networks
– Saul
- 1997
|
|
