Time-delay neural networks: Representation and induction of finite state machines (1997) [11 citations — 5 self]
|
Popular Tags
No tags have been applied to this document.
BibTeX | Add To MetaCart
@ARTICLE{Clouse97time-delayneural,author = {Daniel S. Clouse and C. Lee Giles and Bill G. Horne and Garrison W. Cottrell and D. S. Clouse},
title = {Time-delay neural networks: Representation and induction of finite state machines},
journal = {IEEE Transactions on Neural Networks},
year = {1997},
volume = {8}
}
Years of Citing Articles
Abstract:
In this work, we characterize and contrast the capabilities of the general class of time-delay neural networks (TDNN), with input delay neural networks (IDNN), the subclass of TDNNs with delays limited to the inputs. Each class of networks is capable of representing the same set of languages, those embodied by the definite memory machines (DMM), a subclass of finite state machines. We demonstrate the close affinity between TDNNs and DMM languages by learning a very large DMM (2048 states) using only a few training examples. Even though both architectures are capable of representing the same class of languages, they have distinguishable learning biases. Intuition suggests that general TDNNs which include delays in hidden layers should perform well, compared to IDNNs, on problems in which the output can be expressed as a function on narrow input windows which repeat in time. On the other hand, these general TDNNs should perform poorly when the input windows are wide, or there is little repetition. We confirm these hypotheses via a set of simulations and statistical analysis.
Citations
| 2140 | Learning Internal Representations by Error Propagation – Rumelhart, Hinton, et al. - 1986 |
| 263 | Phoneme Recognition Using Time-Delay Neural Networks – Waibel, Hanazawa, et al. - 1989 |
| 254 | Inductive inference: theory and methods – Angluin, Smith - 1983 |
| 254 | Switching and Finite Automata Theory – Kohavi - 1978 |
| 150 | A time-delay neural network architecture for isolated word recognition – Lang, Waibel, et al. - 1990 |
| 58 | Time series prediction by using a connectionist network with internal delay lines – Wan - 1994 |
| 19 | Learning a class of large finite state machines with a recurrent neural network – Giles, Horne, et al. - 1995 |
| 15 | Shift Register Sequences, Aegean – Golomb - 1981 |
| 14 | On the node complexity of neural networks – Horne, Hush - 1994 |
| 1 | Grammatical Inference and Applications – Carrasco, Oncina - 1994 |
| 1 | Colin de la Higuera, Grammatical Inference: Learning Syntax from Sentences – Miclet - 1996 |
