A paradox of neural encoders and decoders, or, why don't we talk backwards (1999) [5 citations — 2 self]
|
Abstract:
Abstract. We develop a new framework for studying the biases that recurrent neural networks bring to language processing tasks. A semantic concept represented by a point in Euclidian space is translated into a symbol sequence by an encoder network. This sequence is then fed to a decoder network which attempts to translate it back to the original concept. We show how a pair of recurrent networks acting as encoder and decoder can develop their own symbolic language that is serially transmitted between them either forwards or backwards. The encoder and decoder bring different constraints to the task, and these early results indicate that the conflicting nature of these constraints may be reflected in the language that ultimately emerges, providing important clues to the structure of human languages. 1
Citations
| 637 | Aspects of the Theory of Syntax – Chomsky - 1965 |
| 624 | On computable numbers, with an application to the Entscheidungsproblem – Turing - 1936 |
| 221 | Learning and development in neural networks: The importance of starting small – Elman - 1993 |
| 213 | Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning 7 – Elman - 1991 |
| 72 | Toward a connectionist model of recursion in human linguistic performance – Christiansen, Chater - 1999 |
| 45 | Dynamical recognizers: Real-time language recognition by analog computers – Moore - 1998 |
| 37 | S.: Natural language grammatical inference with recurrent neural networks – Lawrence, Giles, et al. |
| 28 | Learning and morphological change – Hare, Elman - 1994 |
| 26 | Fitness and the selective adaptation of language – Kirby - 1998 |
| 14 | A PDP approach to processing center-embedded sentences – Weckerly, Elman - 1992 |
