Download:
|
by Fakultat Fur Informatik, Jurgen Schmidhuber
ftp://ftp.idsia.ch/pub/juergen/ica99.ps.gz
Add To MetaCart
Abstract:
Low-complexity coding and decoding (Lococode), a novel approach to sensory coding, trains autoassociators (AAs) by Flat Minimum Search (FMS), a recent general method for finding low-complexity networks with high generalization capability. FMS works by minimizing both training error and required weight precision. We find that as a by-product Lococode separates nonlinear superpositions of sources without knowing their number. Assuming that the input data can be reduced to few simple causes (this is often the case with visual data), according to our theoretical analysis the hidden layer of an FMS-trained AA tends to code each input by a sparse code based on as few simple, independent features as possible. In experiments Lococode extracts optimal codes for difficult, nonlinear versions of the "noisy bars " benchmark problem, while traditional ICA and PCA do not.
Citations
|
808
|
Independent component analysis, a new concept
– Comon
- 1994
|
|
688
|
An information-maximization approach to blind separation and blind deconvolution
– Bell, Sejnowski
- 1995
|
|
419
|
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
– Olshausen, Field
|
|
372
|
A new learning algorithm for blind signal separation
– Amari, Cichocki, et al.
- 1996
|
|
345
|
Blind separation of sources, Part I: an adaptive algorithm based on a neuromimetic architecture
– Jutten, Hérault
- 1991
|
|
269
|
What is the Goal of Sensory Coding
– Field
- 1999
|
|
264
|
Blind beamforming for non gaussian signals
– Cardoso, Souloumiac
- 1993
|
|
182
|
The wake-sleep algorithm for unsupervised neural networks
– Hinton, Dayan, et al.
- 1995
|
|
122
|
Nonlinear neuron in the low noise limit: A factorial code maximizes information transfer
– Nadal, Parga
- 1994
|
|
111
|
Generative models for discovering sparse distributed representations
– Hinton, Ghahramani
- 1997
|
|
57
|
Competition and multiple cause models
– Dayan, Zemel
- 1995
|
|
47
|
Finding minimum entropy codes
– Barlow, Kaushal, et al.
- 1989
|
|
43
|
Learning factorial codes by predictability minimization
– Schmidhuber
- 1992
|
|
35
|
Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures
– Deco, Brauer
- 1995
|
|
35
|
Inferring sparse, overcomplete image codes using an efficient coding framework
– Lewicki, Olshausen
- 1998
|
|
33
|
Separation of independent signals using time-delayed correlations,” Phys
– Molgedey, Schuster
- 1994
|
|
29
|
Sparse coding in the primate cortex
– Foldiak, Young
- 1995
|
|
28
|
Flat minima
– Hochreiter, Schmidhuber
- 1997
|
|
27
|
Developing population codes by minimizing description length
– Zemel, Hinton
- 1994
|
|
19
|
Semilinear predictability minimization produces well-known feature detectors
– Schmidhuber, Eldracher, et al.
- 1996
|
|
13
|
Discovering discrete distributed representations with iterative competitive learning
– Mozer
- 1991
|
|
9
|
Simplifying nets by discovering flat minima
– Hochreiter, Schmidhuber
- 1995
|
|
5
|
Simplifying neural nets by discovering flat minima
– Hochreiter, Schmidhuber
- 1995
|
