Results 1  10
of
58
Gradient calculation for dynamic recurrent neural networks: a survey
 IEEE Transactions on Neural Networks
, 1995
"... Abstract  We survey learning algorithms for recurrent neural networks with hidden units, and put the various techniques into a common framework. We discuss xedpoint learning algorithms, namely recurrent backpropagation and deterministic Boltzmann Machines, and non xedpoint algorithms, namely backp ..."
Abstract

Cited by 181 (3 self)
 Add to MetaCart
(Show Context)
Abstract  We survey learning algorithms for recurrent neural networks with hidden units, and put the various techniques into a common framework. We discuss xedpoint learning algorithms, namely recurrent backpropagation and deterministic Boltzmann Machines, and non xedpoint algorithms, namely backpropagation through time, Elman's history cuto, and Jordan's output feedback architecture. Forward propagation, an online technique that uses adjoint equations, and variations thereof, are also discussed. In many cases, the uni ed presentation leads to generalizations of various sorts. We discuss advantages and disadvantages of temporally continuous neural networks in contrast to clocked ones, continue with some \tricks of the trade" for training, using, and simulating continuous time and recurrent neural networks. We present somesimulations, and at the end, address issues of computational complexity and learning speed.
The calculi of emergence: Computation, dynamics, and induction
 PHYSICA D
, 1994
"... ..."
(Show Context)
Computation at the onset of chaos
 The Santa Fe Institute, Westview
, 1988
"... Computation at levels beyond storage and transmission of information appears in physical systems at phase transitions. We investigate this phenomenon using minimal computational models of dynamical systems that undergo a transition to chaos as a function of a nonlinearity parameter. For perioddoubl ..."
Abstract

Cited by 101 (19 self)
 Add to MetaCart
Computation at levels beyond storage and transmission of information appears in physical systems at phase transitions. We investigate this phenomenon using minimal computational models of dynamical systems that undergo a transition to chaos as a function of a nonlinearity parameter. For perioddoubling and bandmerging cascades, we derive expressions for the entropy, the interdependence ofmachine complexity and entropy, and the latent complexity of the transition to chaos. At the transition deterministic finite automaton models diverge in size. Although there is no regular or contextfree Chomsky grammar in this case, we give finite descriptions at the higher computational level of contextfree Lindenmayer systems. We construct a restricted indexed contextfree grammar and its associated oneway nondeterministic nested stack automaton for the cascade limit language. This analysis of a family of dynamical systems suggests a complexity theoretic description of phase transitions based on the informational diversity and computational complexity of observed data that is independent of particular system control parameters. The approach gives a much more refined picture of the architecture of critical states than is available via
State space reconstruction in the presence of noise
 Physica D: Nonlinear Phenomena
, 1991
"... SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa Fe Institute. We accept papers intended for publication in peerreviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for pap ..."
Abstract

Cited by 91 (3 self)
 Add to MetaCart
(Show Context)
SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa Fe Institute. We accept papers intended for publication in peerreviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our external faculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, or funded by an SFI grant. ©NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensure timely distribution of the scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the author(s). It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may be reposted only with the explicit permission of the copyright holder. www.santafe.edu
Computational mechanics: Pattern and prediction, structure and simplicity
 Journal of Statistical Physics
, 1999
"... Computational mechanics, an approach to structural complexity, defines a process’s causal states and gives a procedure for finding them. We show that the causalstate representation—an Emachine—is the minimal one consistent with ..."
Abstract

Cited by 68 (13 self)
 Add to MetaCart
(Show Context)
Computational mechanics, an approach to structural complexity, defines a process’s causal states and gives a procedure for finding them. We show that the causalstate representation—an Emachine—is the minimal one consistent with
Is Anything Ever New? Considering Emergence
 IN
, 1994
"... This brief essay reviews an approach to defining and then detecting the emergence of complexity in nonlinear processes. It is, in fact, a synopsis of Reference [1] that leaves out the technical details in an attempt to clarify the motivations behind the approach. The central puzzle ..."
Abstract

Cited by 56 (8 self)
 Add to MetaCart
This brief essay reviews an approach to defining and then detecting the emergence of complexity in nonlinear processes. It is, in fact, a synopsis of Reference [1] that leaves out the technical details in an attempt to clarify the motivations behind the approach. The central puzzle
Information and Its Metric
 Nonlinear Structures in Physical Systems – Pattern Formation, Chaos and Waves
, 1990
"... INTRODUCTION This brief note introduces a measure of distance between information sources. It demonstrates that the space of information sources has much topological structure which heretofore has not been utilized directly in applications of information theory or in the study of complex systems. T ..."
Abstract

Cited by 42 (3 self)
 Add to MetaCart
(Show Context)
INTRODUCTION This brief note introduces a measure of distance between information sources. It demonstrates that the space of information sources has much topological structure which heretofore has not been utilized directly in applications of information theory or in the study of complex systems. The space of information sources is, in fact, a metric space to which a geometric picture can be ascribed. Furthermore, if one considers the elemental events from the information source to be experimental measurements, then the logic of inference is described by a metric lattice. Although this note is restricted to a presentation of formal details and to their geometric interpretation, a few motivational remarks are in order. There are four aspects of the use of information theory in the physics of complex systems that suggest a need for a better understanding of its formal structure and of the concept of information itself. Briefly, these are the observational theory of chaotic syste
Local Dynamic Modeling with SelfOrganizing Maps and Applications to Nonlinear System Identification and Control
 Proceedings of the IEEE
, 1998
"... The technique of local linear models is appealing for modeling complex time series due to the weak assumptions required and its intrinsic simplicity. Here, instead of deriving the local models from the data, we propose to estimate them directly from the weights of a self organizing map (SOM), which ..."
Abstract

Cited by 40 (7 self)
 Add to MetaCart
(Show Context)
The technique of local linear models is appealing for modeling complex time series due to the weak assumptions required and its intrinsic simplicity. Here, instead of deriving the local models from the data, we propose to estimate them directly from the weights of a self organizing map (SOM), which functions as a dynamicpreserving model of the dynamics. We introduce one modification to the Kohonen learning to ensure good representation of the dynamics and use weighted least squares to ensure continuity among the local models. The proposed scheme is tested using synthetic chaotic time series and real world data. The practicality of the method is illustrated in the identification and control of the NASA Langley wind tunnel during aerodynamic tests of model aircrafts. Modeling the dynamics with a SOM leads to a predictive multiple model control strategy (PMMC). Comparison of the new controller against the existing controller in test runs shows the superiority of our method. 1. Introducti...
The Maintenance of Uncertainty
 in Control Systems
, 1997
"... It is important to remain uncertain, of observation, model and law. For the Fermi Summer School, Criticisms Requested email : lenny@maths.ox.ac.uk, Contents 1 ..."
Abstract

Cited by 40 (8 self)
 Add to MetaCart
(Show Context)
It is important to remain uncertain, of observation, model and law. For the Fermi Summer School, Criticisms Requested email : lenny@maths.ox.ac.uk, Contents 1