E. Goles, S. Martnez, Exponential transient classes of symmetric neural networks for synchronous and sequential updating, Complex Systems 3 (1989) 589-597.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....interconnection weight matrix and a saturated linear activation function that simulates an (n 1) bit binary counter and thus produces a sequence of 2 1 well controlled oscillations before it converges. The original idea for a corresponding discrete time symmetric counter network stems from [4]. Besides suggesting some caution in applying neural networks to optimization problems, this provides to our knowledge the rst known example of a continuous time, Liapunov function controlled dynamical system with an exponential transient period. Such an Although in fact the dynamics of this ....

....Counter A continuous time Hop eld system C n of dimension m = 6n 1 will now be constructed which simulates an (n 1) bit binary counter, and thus has a transient period that is exponential in the parameter m. The original idea for a corresponding discrete time counter network stems from [4]. In our simulation, the binary states 0 and 1 of the counter will be represented by excitations (2) of the corresponding real valued units in C that are either below the lower saturation threshold of 0 or above the upper saturation threshold of 1, respectively, for activation function (3) For ....

E. Goles, S. Martnez, Exponential transient classes of symmetric neural networks for synchronous and sequential updating, Complex Systems 3 (1989) 589-597.

....computation in the course of simulation possibly with some constant time overhead per each original update. The idea behind this simulation is that each directed edge is implemented by a small symmetric subnetwork which receives energy support from a symmetric clock subnetwork (a binary counter) [11] in order to propagate a signal in the right direction. This result may also be interpreted within the context of infinite families of neural networks which, each for one input length, can be exploited for universal computations (similarly as circuit families) Thus the infinite sequences of ....

....case bounds while the average case analysis can be found in [17] Obviously, there are exactly 2 n different states in a network with n binary neurons which yields trivial 2 n upper bound on the convergence time in symmetric networks of size n. On the other hand, the symmetric clock network [11] which is used in the proof of Theorem 1 represents an explicit example of a Hopfield net whose convergence time is exponential with respect to n. Namely, this gives Omega (2 n=3 ) lower bound on the convergence time of Hopfield nets since the respective (k 1) bit binary counter requires n = ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3, 589--597, 1989.

....energize the computation for a sufficiently long time. 5. 1 A Simulated Binary Counter By induction on n we will now shortly sketch the construction of a binary state symmetric clock network C n of size s n = 3n 1 units, which simulates an (n 1) bit binary counter under fully parallel updates [23, 24]. Assume that all the states in the clock are initially zero. The induction starts with a network C 0 of order 0 containing only a single unit c 0 with zero bias w(0; c 0 ) 0, which represents the least significant counter bit. As assumed, neuron c 0 is initially passive (its state is 0) ....

.... explicit example of a Hopfield net whose convergence time, i.e. the number of discrete time updates before the network converges, is exponential [25] In particular, this yields a 3 s=3 ) lower bound on the convergence time of binary Hopfield nets with s neurons working in a fully parallel mode [23, 24]. An asynchronous implementation of the binary counter by a symmetric network was designed [26] witnessing a corresponding exponential convergence time lower bound u s=8 ) also for sequential updates [27] On the other hand, in Hopfield nets of s binary neurons a trivial 2 s upper bound holds ....

E. Goles-Chacc and S. Martnez, Exponential transient classes of symmetric neural networks for synchronous and sequential updating, Complex Systems 3 (6) (1989) 589--597.

....Thus, in particular, networks with polynomially bounded weights converge in polynomial time. On the other hand, networks with exponentially large weights may indeed require an exponential time to converge, as was first shown in [31] for synchronous updates (a simplified construction appears in [29]) and in [34] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [31] or [29] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any asynchronous update rule was demonstrated in ....

....weights may indeed require an exponential time to converge, as was first shown in [31] for synchronous updates (a simplified construction appears in [29] and in [34] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [31] or [29] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any asynchronous update rule was demonstrated in [33] This result is now also known to follow from the more general theory of PLS completeness for local optimization problems [87] Related ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3 (1989), 589--597.

....for both sequential [43] and parallel [40] modes. This is witnessed e.g. by symmetric, sequential or parallel implementations of a binary counter that traverses most of the network space before it converges in time i s=8 ) asynchronous sequential updates [42] or s=3 ) fully parallel steps [38, 39]. For the average case, on the other hand, a very fast convergence O(log log s) of binary 11 Hop eld nets can be shown under reasonable assumptions [61] However, the previous bounds do not take into account the size of the weights. An upper bound of O(W ) on the convergence time of binary ....

E. Goles-Chacc and S. Martnez, Exponential transient classes of symmetric neural networks for synchronous and sequential updating, Complex Systems 3 (6) (1989) 589-597.

....subsystem of 10 continuous time variables. A similar, but considerably simpler, construction was used in [16] to prove the computational equivalence of symmetrically interconnected and arbitrary threshold gate networks. The original idea for the discrete time clock network used in [16] stems from [6]. The continuous time clock is already by itself of some interest from a dynamical systems perspective, because it provides to our knowledge the rst known example of a continuous time Liapunov system whose convergence time grows exponentially in the system dimension. It is quite easy to see ....

E. Goles and S. Martnez. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3:589{ 597, 1989.

....1 An interesting question here is how efficient is the representation of finite automata as neural nets. It was shown recently in [3] that representing an automaton of n states may require Omega ( n log n) 1=3 ) neurons in the worst case. updates (a simplified construction appears in [23]) and in [28] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [25] or [23] by a fairly simple construction. Finally, a network requiring exponential time for stabilization under any asynchronous update rule was demonstrated ....

....representing an automaton of n states may require Omega ( n log n) 1=3 ) neurons in the worst case. updates (a simplified construction appears in [23] and in [28] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [25] or [23] by a fairly simple construction. Finally, a network requiring exponential time for stabilization under any asynchronous update rule was demonstrated in [27] Related work appears in [8, 24] Somewhat surprisingly, the very constrained convergence behavior of symmetric nets is not reflected in ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3 (1989), 589--597.

....bounded in the number of units n, then the network converges in polynomial time. Conversely, it can be shown that networks with exponentially large weights may indeed require an exponential time to converge. This was rst shown in [17] for synchronous updates (a simpli ed construction appears in [14]) and in [19] for a particular sequential update rule. For a di erent sequential rule the result actually follows already from [17] or [14] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any sequential update rule was demonstrated in [18] ....

....large weights may indeed require an exponential time to converge. This was rst shown in [17] for synchronous updates (a simpli ed construction appears in [14] and in [19] for a particular sequential update rule. For a di erent sequential rule the result actually follows already from [17] or [14] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any sequential update rule was demonstrated in [18] 2 3. FINITE BINARY STATE NETWORKS It has been known since the early work of McCulloch and Pitts [32] and Kleene [25] that nite ....

[Article contains additional citation context not shown here]

Goles, E., Martnez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3 (1989), 589-597.

....of the simulation possibly with some constant time overhead per each original update. The idea behind this simulation is that each asymmetric edge to be simulated is implemented by a small symmetric subnetwork which receives energy support from a symmetric clock subnetwork (a binary counter) (Goles and Mart nez, 1989) in order to propagate a signal in the correct direction. In the context of in nite families of neural networks, which contain one network for each input length (a similar model is used in the study of Boolean circuit complexity (Wegener, 1987) this simulation result implies that in nite ....

....only worst case bounds; an average case analysis can be found in (Koml os and Paturi, 1988) Since a network with n binary neurons has 2 n di erent states, a trivial 2 n upper bound on the convergence time in symmetric networks of size n holds. On the other hand, the symmetric clock network (Goles and Mart nez, 1989) which is used in the proof of Theorem 1 11 provides an explicit example of a Hop eld net whose convergence time is exponential with respect to n. More precisely, this network yields a 4 n=3 ) lower bound on the convergence time of Hop eld nets, since a (k 1) bit binary counter can be ....

Goles, E. and Martnez, S. (1989). Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3:589-597.

....in particular, symmetric networks with polynomially bounded weights converge in polynomial time. On the other hand, networks with exponentially large weights may indeed require an exponential time to converge, as was first shown in [11] for synchronous updates (a simplified construction appears in [9]) and in [13] for a particular asynchronous update rule. A network requiring exponential time to converge under any asynchronous update rule was demonstrated in [12] Related work appears in [5, 10] In this paper, we prove that despite their apparently very constrained behavior also symmetric ....

....in section 4, is to start with the simulation of space bounded Turing machines by asymmetric nets, and then replace each of the asymmetric edges by a sequence of symmetric edges whose behavior is sequenced by clock pulses. The appropriate clock can be obtained from, e.g. the construction in [9] of a network that takes an exponential number of steps to converge. Of course, this clock network does not run forever (in this case it is not sufficient to have a network that simply oscillates between two states) but the exponentially long sequence of pulses it generates is sufficient to ....

[Article contains additional citation context not shown here]

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3 (1989), 589--597.

.... ofsymmetric and convergent asymmetric discrete time binary networks 1 , and the simulation of discrete time networks by asymmetric continuous time networks in [14] The original idea for the discrete time clock network used in [12] and on which our current construction is based, stems from [4]. A general survey of topics in continuous time computation is presented in [13] 1 Our present construction can actually also be used to improve the discrete time simulation in [12] which requires a symmetric network of n 2 ) units to simulate a convergent asymmetric network of size n. ....

Goles, E., Martnez, S. Exponential transient classes of symmetric neural networks for syn- chronous and sequential updating. Complex Systems, 3, 589-597, 1989.

....of the simulation possibly with some constant time overhead per each original update. The idea behind this simulation is that each asymmetric edge to be simulated is implemented by a small symmetric subnetwork which receives energy support from a symmetric clock subnetwork (a binary counter) [17] in order to propagate a signal in the correct direction. In the context of in nite families of neural networks, which contain one network for each input length (a similar model is used in the study of Boolean circuit complexity [48] this simulation result implies that in nite sequences of ....

....We shall consider 7 only worst case bounds; an average case analysis can be found in [26] Since a network with n binary neurons has 2 n di erent states, a trivial 2 n upper bound on the convergence time in symmetric networks of size n holds. On the other hand, the symmetric clock network [17] which is used in the proof of Theorem 1 provides an explicit example of a Hop eld net whose convergence time is exponential with respect to n. More precisely, this network yields a 3 n=3 ) lower bound on the convergence time of Hop eld nets, since a (k 1) bit binary counter can be implemented ....

E. Goles and S. Martnez. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3:589{ 597, 1989.

....i.e. they compute the complexity class PSPACE poly or P poly when polynomial weights are considered. The idea behind this simulation is that each directed edge is implemented by a small symmetric subnetwork which receives an energy support from a symmetric clock subnetwork (a binary counter) [11] in order to propagate a signal in the right direction. In this section the construction from [20] will be improved by reducing the number of neurons in the simulating symmetric network to the linear size 6n 2 which is asymptotically optimal. This is achieved by simulating the neurons (instead ....

....bounds while the average case analysis can be found in [17] Obviously, there are exactly 2 n different states in a network with n binary neurons which yields trivial 2 n upper bound on the convergence time in symmetric networks of the size n. On the other hand, the symmetric clock network [11] which was used in the proof of Theorem 1 represents an explicit example of a Hopfield net whose convergence time is exponential with respect to n. Namely, this gives Omega Gamma n=3 ) lower bound on the convergence time of Hopfield nets since the respective (k 1) bit binary counter requires ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3, 589--597, 1989.

.... equivalence of symmetric and asymmetric discrete time binary networks 3 , and the simulation of discrete time networks by asymmetric continuous time networks in [12] The original idea for the discretetime clock network used in [10] and on which our current construction is based, stems from [3]. A general survey of topics in continuous time computation is presented in [11] As pointed out in [10] polynomial size increasing sequences of discrete networks are computationally equivalent to (nonuniform) polynomially space bounded Turing machines (more precisely, they compute the complexity ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3, 589--597, 1989.

....stops. Note that a convergent synchronous computation on n units must terminate within 2 n steps, because otherwise the network repeats a configuration and goes into a cycle. Such a clock can be obtained from, e.g. a symmetric exponential transient network designed by Goles and Mart inez [1]. The first two stages in the construction of this network are presented in Figure 6. The idea here is that the n units in the upper row implement a binary counter, counting from all 0 s to all 1 s (in the figure, the unit corresponding to the least significant bit is to the right) For each ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3 (1989), 589--597.

.... equivalence of symmetric and asymmetric discrete time binary networks 1 , and the simulation of discrete time networks by asymmetric continuous time networks in [12] The original idea for the discrete time clock network used in [10] and on which our current construction is based, stems from [3]. A general survey of topics in continuous time computation is presented in [11] As pointed out in [10] polynomial size increasing sequences of discrete networks are computationally equivalent to (nonuniform) polynomially space bounded Turing machines (more precisely, they compute the complexity ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3, 589--597, 1989.

....in particular, symmetric networks with polynomially bounded weights converge in polynomial time. On the other hand, networks with exponentially large weights may indeed require an exponential time to converge, as was first shown in [12] for synchronous updates (a simplified construction appears in [10]) and in [14] for a particular asynchronous update rule. A network requiring exponential time to converge under any asynchronous update rule was demonstrated in [13] Related work appears in [6, 11] In this paper, we prove that despite their apparently very constrained behavior also symmetric ....

....in section 4, is to start with the simulation of space bounded Turing machines by asymmetric nets, and then replace each of the asymmetric edges by a sequence of symmetric edges whose behavior is sequenced by clock pulses. The appropriate clock can be obtained from, e.g. the construction in [10] of a symmetric network that takes an exponential number of steps to converge. Obviously, such a clock network cannot run forever (in this case it is not sufficient to have a network that simply oscillates between two states) but the exponentially long sequence of pulses it generates is ....

[Article contains additional citation context not shown here]

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3 (1989), 589--597.

....1 2 1 Gamma1 Figure 3: A three bit fully parallel binary counter network. standard local improvement algorithm requires exponential time in the worst case. For more details on this approach, see Section 5.2. Here we present two explicit constructions: first a network due to Goles and Mart inez [29, 30] that requires an exponential number of fully parallel steps to converge, and then a more involved construction by Haken [32] of a network whose convergence time is exponential also in sequential operation 3 . Let us first consider fully parallel updates [29] For a given n, we construct a ....

....due to Goles and Mart inez [29, 30] that requires an exponential number of fully parallel steps to converge, and then a more involved construction by Haken [32] of a network whose convergence time is exponential also in sequential operation 3 . Let us first consider fully parallel updates [29]. For a given n, we construct a network of size linear in n that functions as an n bit binary counter, and thus takes time at least 2 n to converge. The construction is most easily presented in terms of binary (0 1 valued) nodes, but the networks can of course be translated to the bipolar ....

[Article contains additional citation context not shown here]

E. Goles and S. Mart'inez. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3:589--597, 1989.

....Thus, in particular, networks with polynomially bounded weights converge in polynomial time. On the other hand, networks with exponentially large weights may indeed require an exponential time to converge, as was first shown in [31] for synchronous updates (a simplified construction appears in [29]) and in [34] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [31] or [29] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any asynchronous update rule was demonstrated in ....

....weights may indeed require an exponential time to converge, as was first shown in [31] for synchronous updates (a simplified construction appears in [29] and in [34] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [31] or [29] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any asynchronous update rule was demonstrated in [33] This result is now also known to follow from the more general theory of PLS completeness for local optimization problems [87] Related ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3 (1989), 589--597.

....other hand, networks with exponentially large weights may indeed require an exponential time to converge, as was first shown by Goles and Olivos (1981) for synchronous updates, and by Haken and Luby (1988) for a particular asynchronous update rule. The former construction was later simplified by Goles and Mart inez (1989). A network requiring exponential time to converge under an arbitrary asynchronous update order was first demonstrated by Haken (1989) The existence of networks with exponentially long asynchronous transients is now known to follow also from the general theory of local search for optimization ....

....with the simulation of space bounded Turing machines by asymmetric nets, and then replace each of the asymmetric edges by a sequence of symmetric edges whose behavior is sequenced by clock pulses. The appropriate clock can be obtained from, e.g. the symmetric exponential transient network by Goles and Mart inez (1989). Obviously, such a clock network cannot run forever (in this case it is not sufficient to have a network that simply oscillates between two states) but nevertheless the sequence of pulses it generates is long enough to simulate a polynomially space bounded computation or, in the case of ....

[Article contains additional citation context not shown here]

Goles, E., and Mart'inez, S. 1989. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3, 589--597.

....otherwise interesting constructions of recurrent threshold logic networks (or, more generally, automata networks) is their use of a global synchronizing mechanism. It is commonly assumed that either the computational units in the network update their states fully synchronously in parallel (e.g. [8, 10, 11, 16, 24]) or there is some a priori imposed sequential update order (e.g. 2, 25] or some intermediate form of the two applies (e.g. 7] Such global timing constraints are clearly not consonant with the otherwise distributed nature of the model, where the behavior of each unit in other respects ....

....Department of Computer Science, University of Helsinki, Finland. E mail: orponen igi.tu graz.ac.at available information. On the other hand, experience has shown that programming such networks without any assumptions on synchronization is rather awkward. For instance, Goles et al. constructed in [8, 11] symmetric threshold logic networks whose transient times under parallel updates are exponential in the number of units in the network. Tchuente [25] and, independently, Bruck and Goodman [2] then came up with a simple method to simulate parallel updates by updates that are performed in a cyclic ....

Goles, E., Mart'inez, S. Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems 3 (1989), 589--597.

No context found.

E. Goles, S. Martnez, Exponential transient classes of symmetric neural networks for synchronous and sequential updating, Complex Systems 3 (1989) 589-597.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC