M.W. Hirsch. Saturation at high gain in discrete time recurrent networks. Neural Networks, 7(3):449-- 453, 1994.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....is typically a hyperbolic tangent. Of interest in these types of networks is the nature of fixed points of (13) In particular, we are concerned with the number of stable fixed points, and their location in the state space. Using the results in this paper we demonstrate a result similar to one in [2], which shows that as the weight matrix becomes large, the stable fixed points of (13) cannot be located near the center of the state space. Our results apply to weight matrices that are either symmetric with zeros along the diagonal, or skew symmetric. The stability of a fixed point, x, of (13) ....

M.W. Hirsch. Saturation at high gain in discrete time recurrent networks. Neural Networks, 7(3):449--453, 1994.

....There is no known analog of this for stable periodic orbits of discrete time networks. The only known stability result asserts that for a broad class of discrete time networks where all output neurons are either self inhibiting or self exciting, outputs at stable fixed points saturate at high gain [Hirsch 1994]. Our proof of stability of an internal DFA state representation establishes such a result for a special case of discrete time recurrent networks. Our method is an alternative to an algorithm for constructing DFAs in recurrent networks with first order weights proposed by Frasconi et al. 1991; ....

HIRSCH, M. 1994. Saturation at high gain in discrete time recurrent networks. Neural Netw. 7,3, 449 -- 453.

....to the state where we have started) As reported in [6] 14] and [15] loops and cycles associated with an input symbol x are usually represented as attractive fixed points and periodic orbits respectively of the underlying dynamical system corresponding to the input x. It was proved by Hirsh [11], that when all the weights in a recurrent network with exclusively self exciting (or exclusively self inhibiting) neurons are multiplied by larger and larger positive number (neural gain) attractive fixed points tend to saturated activation values. In section 6, for an N neuron recurrent ....

.... W 1=N in case of B = 0 and A = 1. The dashed and solid lines correspond to functions 0:5 Gamma Delta(v) and 0:5 Delta(v) respectively. For a particular value of v, H is a hypercube with the side centered at 0:5 and spanned between the two curves. 6. 1 The effect of growing neural gain In [11], Hirsh studies recurrent networks with nondecreasing activation functions f from a broader class than the sigmoid shaped class considered here. Activation functions can differ from neuron to neuron. He shows that attractive fixed point coordinates corresponding to neurons with steadily ....

M.W. Hirsch. Saturation at high gain in discrete time recurrent networks. Neural Networks, 7(3):449--453, 1994.

....There is no known analog of this for stable periodic orbits of discrete time networks. The only known stability result asserts that for a broad class of discrete time networks where all output neurons are either self inhibiting or self exciting, outputs at stable fixed points saturate at high gain [15]. Our proof of stability of an internal DFA state representation establishes such a result for a special case of discrete time recurrent networks. Our method is an alternative to an algorithm for constructing DFAs in recurrent networks with first order weights proposed by Frasconi et al. 6, 7] A ....

M. Hirsch, "Saturation at high gain in discrete time recurrent networks," Neural Networks, vol. 7, no. 3, pp. 449--453, 1994.

....There is no known analog of this for stable periodic orbits of discrete time networks. The only known stability result asserts that for a broad class of discrete time networks where all output neurons are either self inhibiting or self exciting, outputs at stable fixed points saturate at high gain [16]. Our proof of stability of an internal DFA state representation establishes such a result for a special case of discrete time recurrent networks. Our method is an alternative to an algorithm for constructing DFAs in recurrent networks with first order weights proposed by Frasconi et al. 7, 5] A ....

M. Hirsch, "Saturation at high gain in discrete time recurrent networks," Neural Networks, vol. 7, no. 3, pp. 449--453, 1994.

....as well as the use of simple clustering technique introduced in [15] are supported. 6 RNN as a Collection of Dynamical Systems RNNs can be viewed as discrete time DSs. Literature dealing with the relationship between RNNs and DSs is quite rich: 18] 3] 14] 22] 24] 33] 34] 32] 2] [19], 5] for example. However, as it has been already mentioned, the task of complete understanding of the global dynamical behaviour of a given DS is not at all an easy one. In [34] it is shown that networks with just two recurrent neurons can exhibit chaos and hence the asymptotic network dynamical ....

.... of vertices of unit square, where = q (0:5 Gamma Delta(ff) 2 (0:5 Gamma Delta(ffi) 2 : The tendency of attractive fixed points in discrete time RNNs with exclusively self exciting recurrent neurons to move towards saturation values as neural gain grows is also discussed in [19]. So far, we have confined the areas of the network state space (0; 1) 2 where (under some assumptions on weights) fixed points of (14) of particular stability types can lie. In the following, it will be shown that those regions correspond to monotonicity intervals of functions defining fixed ....

M.W. Hirsch. Saturation at high gain in discrete time recurrent networks. Neural Networks, 7(3):449--453, 1994.

....component is a hyperbolic tangent. Of interest in these types of networks is the nature of fixed points of (13) In particular, we are concerned with the number of stable fixed points, and their location in the state space. Using the results in this paper we demonstrate a result similar to one in [2], which shows that as the weight matrix becomes large, the stable fixed points of (13) must move away from the center of the state space. Our results apply to weight matrices that are either symmetric with zeros along the diagonal, or skew symmetric. The stability of a fixed point, x, of (13) ....

M.W. Hirsch. Saturation at high gain in discrete time recurrent networks. Neural Networks, 7(3):449--453, 1994.

....regions corresponding to stability types of the fixed points. This is done by first exploring the space of derivatives of the sigmoid transfer function with respect to the weighted sum of neuron inputs. Then, the structure is transformed into the space of neuron activations. It was proved by Hirsh [13], that when all the weights in a recurrent network with exclusively 1 In such a case the recurrent network is shown to have only one fixed point and no genuine periodic orbits (of period greater than one) self exciting (or exclusively self inhibiting) neurons are multiplied by larger and ....

....be also appropriately increased so as to compensate for the increase in d so that the bended part of f d;c does not move radically to higher values of x. This tendency, in the context of networks with exclusively self exciting (or exclusively selfinhibiting) recurrent neurons, is discussed in [13]. Our result stated in Corollary 1, assumes two neuron recurrent network. It only requires that the neurons have the same mutual interaction pattern (bc 0) and gives a lower bound on the rate of convergence of the attractive fixed points of (2) towards some of the vertices f0; 1g 2 , as the ....

M.W. Hirsch. Saturation at high gain in discrete time recurrent networks. Neural Networks, 7(3):449--453, 1994.

No context found.

M.W. Hirsch. Saturation at high gain in discrete time recurrent networks. Neural Networks, 7(3):449-- 453, 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC