Developing population codes by minimizing description length (1994) [27 citations — 8 self]
|
Popular Tags
No tags have been applied to this document.
BibTeX | Add To MetaCart
@INPROCEEDINGS{Zemel94developingpopulation,author = {Richard S. Zemel and Geoffrey E. Hinton},
title = {Developing population codes by minimizing description length},
booktitle = {Neural Computation},
year = {1994},
pages = {11--18},
publisher = {Morgan Kaufmann}
}
Years of Citing Articles
Abstract:
The Minimum Description Length principle (MDL) can be used to train the hidden units of a neural network to extract a representation that is cheap to describe but nonetheless allows the input to be reconstructed accurately. We show how MDL can be used to develop highly redundant population codes. Each hidden unit has a location in a low-dimensional implicit space. If the hidden unit activities form a bump of a standard shape in this space, they can be cheaply encoded by the center of this bump. So the weights from the input units to the hidden units in an autoencoder are trained to make the activities form a standard bump. The coordinates of the hidden units in the implicit space are also learned, thus allowing flexibility, as the network develops a discontinuous topography when presented with different input classes. Population-coding in a space other than the input enables a network to extract nonlinear
