Generation of desired trajectory behavior using neural networks involves a particularly challenging spario-temporal learning problem due to the need for employing architectures with recurrent connections. This paper introduces a novel solution to the trajectory generation problem, i.e. designing a dynamic system whose terminal behavior emulates a prespecified spatio-temporal pattern independently of its initial conditions. The proposed solution uses a Dynamic Neural Network (DNN), a hybrid architecture that employs a Recurrent Neural Network (RNN) in cascade with a Non-Recurrent Neural Network (NRNN). The RNN is in charge of generating a simple limit cycle while the NRNN is devoted to reshaping the limit cycle into the desired trajectory. The main advantage of this architecture is the simplicity of its training which results from the simplification of the overall training task due to its decomposition into independent spatial and temporal learning subtasks, which in turn permits to reduce the training complexity to that of training a feedforward neural network alone. The solution to the trajectory generation problem presented here involves the synthesis of a very simple RNN which eliminates the need for any type of training of this network and enables all of the required training to be focused on the NRNN. As part of the overall design of the dynamic neural network architecture, a systematic synthesis procedure based on the design of relay control systems is developed for
|
4514
|
Statistical Learning Theory
– Vapnik
- 1998
|
|
800
|
Multilayer feedforward networks are universal approximators
– Hornik, Stinchcombe, et al.
- 1989
|
|
529
|
Approximation by superposition of a sigmoidal function
– Cybenko
- 1989
|
|
431
|
Learning representation by back propagating errors
– Rumelhart, Hinton, et al.
- 1986
|
|
403
|
Neurons with graded responses have collective computational properties like those of two-state neurons
– Hopfield
- 1984
|
|
322
|
A learning algorithm for continually running fully recurrent neural networks
– Williams, Zipser
- 1990
|
|
265
|
On the approximate realization of continuous mappings by neural networks
– Funahashi
- 1989
|
|
210
|
Regularization algorithms for learning that are equivalent to multilayer networks
– Poggio, Girosi
- 1990
|
|
201
|
Learning long-term dependencies with gradient descent is dif�cult
– Bengio, Simard, et al.
- 1994
|
|
174
|
Learning from data, concepts, theory and methods
– Cherkassky, Mulier
- 1998
|
|
159
|
An evolutionary algorithm that constructs recurrent neural networks
– Angeline, Saunders, et al.
- 1993
|
|
143
|
Backpropagation Through Time: What It Does and How to Do it
– Werbos
- 1990
|
|
109
|
Gradient Calculations for Dynamic Recurrent Neural Networks: A Survey
– Pearlmutter
- 1995
|
|
106
|
Generalization of back-propagation to recurrent neural networks
– Pineda
- 1987
|
|
101
|
An e cient gradient-based algorithm for on-line training of recurrent network trajectories
– Williams, Peng
- 1990
|
|
55
|
Neural net algorithms that learn in polynomial time from examples and queries
– Baum
- 1991
|
|
54
|
Discrete-Time Control Systems
– Ogata
- 1995
|
|
52
|
Approximation of dynamical systems by continuous time recurrent neural networks
– Funahashi, Nakamura
- 1993
|
|
39
|
Efficient Parallel learning algorithms for neural networks
– Kramer, Sangiovanni-Vincentelli
- 1988
|
|
29
|
Interpolation and extrapolation of stationary random sequences
– Kolmogorov
- 1941
|
|
14
|
On the Problem of Local Minima in Recurrent Neural Networks
– Bianchini, Gori, et al.
- 1994
|
|
13
|
A fast, compact approximation of the exponential function
– Schraudolph
- 1999
|
|
12
|
Learning a Trajectory Using Adjoint Functions and Teacher Forcing
– Toomarian, Barhen
- 1992
|
|
8
|
A method for improving the real-time recurrent learning algorithm
– Catfolis
- 1993
|
|
8
|
Existence, learning, and replication of periodic motions in recurrent neural networks
– Ruiz, Owens, et al.
- 1998
|
|
6
|
Faster learning for dynamic recurrent backpropagation
– Fang, Sejnowski
- 1990
|
|
6
|
Trajectory production with the adaptive timedelay neural network
– Lin, Dayhoff, et al.
- 1995
|
|
6
|
Learning continuous trajectories in recurrent neural networks with time-dependent weights
– Galicki, Leistritz, et al.
- 1999
|
|
6
|
Recurrent neural-network training by a learning automaton approach for trajectory learning and control system
– Sundareshan, Condarcure
- 1998
|
|
6
|
Relay Control Systems
– Tsypkin
- 1984
|
|
4
|
Learning state-space trajectories in recurrent neural networks
– Pearlmutter
- 1989
|
|
4
|
Efficient training of recurrent neural network with time delays
– Cohen, Saad, et al.
- 1997
|
|
4
|
Existence and learning of oscillations in recurrent neural networks
– Townley, Ilchmann, et al.
- 1999
|
|
3
|
Identification and decentralized adaptive control using dynamical neural networks with application to robotic manipulators
– Karakasoglu, Sudharsanan, et al.
- 1993
|
|
3
|
An Optimal Tracking Neuro-Controller for Nonlinear Dynamic Systems
– Park, Choi, et al.
- 1996
|
|
3
|
Recurrent neural network training with feedforward complexity
– Olurotimi
- 1994
|
|
3
|
Supervised Training of Dynamical Neural Networks for Associative Memory Design and Identification of Nonlinear Maps
– Sudharsanan, Sundareshan
- 1994
|
|
3
|
Periodic Motions, Mapping Ordered Sequences and training of dynamic neural networks to generate continuous and discontinuous trajectories
– Zegers, Sundereshan
- 2000
|
|
3
|
Astrom and B Wittenmark. Adaptive Control
– J
- 1994
|
|
2
|
On-line improvement of speed and tracking performance on repetitive paths
– Boe, Hannaford
- 1998
|
|
2
|
Patterns of dynamic activity and timing in neural network processing
– Dayhoff, Palmadesso, et al.
- 1998
|
|
2
|
Training algorithms for recurrent neural nets that eliminate the need for computation of error gradients with applications to trajectory production problems
– Sundareshan, Wong, et al.
- 1999
|
|
2
|
Automatic Tuning of Simple Regulators
– Astrom, Hagglund
- 1984
|
|
1
|
Fast training of multilayer percepttons
– Verma
- 1997
|
|
1
|
Astrom and Hagglund, "Automatic tuning of simple regulators with specifications on phase and amplitude margins
– J
- 1984
|
|
1
|
Comparison of recurrent neural networks for trajectory generation
– Hagher, Hassoun, et al.
- 1999
|
|
1
|
Some New Results on The Architecture, Training Process, and Estimation Error Bounds for Learning Machines
– Zegers
- 2002
|
