K. Doya. Universality of fully-connected recurrent neural networks. Technical report, University of California, San Diego, 1993.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....networks are thus much more adapted to temporal treatment than the classical feedforward ones which are much more adapted to classification tasks. Moreover, they can deal with non fixpoint learning algorithms and it was proven that these networks are universal approximators of dynamical system [1]. The network has to treat temporal sequences, it follows that the learning equations will be continuous in time and time will appear explicitly. Dynamic neural networks are governed by the following equations : T i dy i dt = Gammay i F (x i ) I i (1) where y i is the state or ....

K. Doya. Universality of fully-connected recurrent neural networks. Technical report, University of California, San Diego, 1993.

....like dynamics, chaos, route to chaos and non fixedpoint attractors will be more and more introduced. Accordingly, if feedforward networks are universal approximators for continuous functions, it has been proved that recurrent neural networks are universal approximators of dynamical systems [17]. The goal of the paper is to provide a strong theoretical basis for modeling and simulating dynamic recurrent neural networks. The analysis of the dynamical behavior will help to understand the capabilities and limitations of these architectures. Our goal will be achieved by studying the effect ....

K. Doya. Universality of fully-connected recurrent neural networks. Technical report, University of California, San Diego, 1993.

....er tilstraekkeligt til at approksimere en vilk#rlig kontinuert funktion. Disse beviser er gennemfrt af Cybenko, se [Haykin 94] for referencer. Beviserne siger dog ikke noget om, hvor mange skjulte enheder, der skal til for en given type funktion. For rekursive netvaerk er det tilsvarende vist ( Doya 93] at et netvaerk med et lag skjulte enheder, lineaere outputs og feedback fra outputtet til de skjulte enheder kan approksimere en vilk#rlig map; en map er en beskrivelse af et dynamisk system ved et saet af differensligninger ( Tsonis 94] Beviset bygger netop p#, at feedforward delen i hvert ....

....gjort det praktisk muligt at anvende fuldt rekursive netvaerk, er der i litteraturen meget f# eksempler p# anvendelser af disse netvaerk. Specielt er det vanskeligt at finde eksempler p# anvendelse til tidsseriepraediktion, hvilket blev klart under litteraturstudiet til dette projektarbejde. I [Doya 93] bliver der ogs# gjort opmaerksom p# litteraturmanglen. Endelig var der ved konferencen ICANN 94 ud af omkring 340 bidrag kun eet ( Guignot 94] der omhandlede fuldt rekursive netvaerk. Det generelle indtryk omkring fuldt rekursive netvaerk er, at mange taler om dem, men kun f# anvender dem i ....

Doya, K. : Universality of Fully-Connected Recurrent Neural Networks. Sendt til IEEE Transactions on Neural Networks. Tilgaengelig via ftp fra Neuroprose : doya.universality.ps.Z (1993).

....learning an input output mapping. It has been proven[Hornik91] that one of the supervised learning algorithms Backpropagation learning can approximate any continuous input output static mapping to any degree of accuracy, by a multilayer neural network, if there are enough hidden neurons. Doya93] and others have extended this result to include temporal patterns, but using recurrent neural architectures. The more restricted Simple Recurrent Network was proven [Sperduti97] such able to simulate any frontier to root automata, while some other recurrent models as cascade correlation networks ....

Doya, K. (1993). Universality of fully connected recurrent neural networks. Technical report. Univ.of California, San Diego. 33

....to simulate the missing atoms through a composition of functions for which it is easy to compute the above derivatives. Since the capability of well sized feedforward neural networks to approximate static mappings [32, 55] and recurrent neural networks to approximate discrete time dynamic systems [31, 39] is well assessed, we know that a proper connective tissue between the above pieces of knowledge exists which allows us to simulate the whole function F. Beside trial and error strategies, we intervene in the choice of the neural architecture in two directions, as will be clear in the application ....

K. Doya, "Universality of fully-connected recurrent neural networks", TR Dept of Biology, Univ of California, San Diego, 1993.

....motivated NNs are supported further by a number of theoretical analyses. Hornik et al. 1991) proved that Backpropagation learning can approximate any continuous input output static mapping to any degree of accuracy, by a multilayer neural network, if there are enough hidden neurons. Doya (1993) extended this result to include temporal patterns using recurrent neural architectures. The more restricted SRNs were proven by Sperduti (1997) able to simulate any frontier to root automaton, while some other recurrent models such as cascade correlation networks and Neural Trees can not. The ....

Doya, K. (1993). Universality of fully connected recurrent neural networks. Technical report. Univ.of California, San Diego.

....Recurrent neural networks are expected to have versatile capabilities for modeling and controlling dynamical systems. From the fact that multi layer neural networks can approximate arbitrary mappings [13] it is easy to show that recurrent neural networks can model arbitrary dynamical systems [3]. Back propagation learning schemes for multi layer feed forward networks have been successfully applied to a wide range of problems. In contrast, since gradient descent learning algorithms for recurrent networks became popular several years ago [19, 5, 18, 25] not many cases have been reported ....

K. Doya. Universality of fully connected recurrent neural networks. (submitted).

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC