A. Haken, Connectionist networks that need exponential time to stabilize, Unpublished manuscript, Department of Computer Science, University of Toronto (1989).  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... before the network converges, is exponential [25] In particular, this yields a 3 s=3 ) lower bound on the convergence time of binary Hopfield nets with s neurons working in a fully parallel mode [23, 24] An asynchronous implementation of the binary counter by a symmetric network was designed [26] witnessing a corresponding exponential convergence time lower bound u s=8 ) also for sequential updates [27] On the other hand, in Hopfield nets of s binary neurons a trivial 2 s upper bound holds since there are only 2 s different network states. Nevertheless, a very fast average case ....

A. Haken, Connectionist networks that need exponential time to stabilize, Unpublished manuscript, Department of Computer Science, University of Toronto (1989).

....eld nets may indeed be exponential for both sequential [44] and parallel [41] modes. This is witnessed e.g. by symmetric, sequential or parallel implementations of a binary counter that traverses most of the network state space before it converges in time m s=8 ) asynchronous sequential updates [43] or s=3 ) fully parallel steps [39] 40] On the other hand, a very fast average case convergence of only O(log log s) parallel update steps can be shown for binary Hop eld nets under reasonable assumptions [63] However, the previous bounds do not take into account the size of the weights. An ....

A. Haken, Connectionist networks that need exponential time to stabilize, Unpublished manuscript, Department of Computer Science, University of Toronto, 1989.

....and in [34] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [31] or [29] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any asynchronous update rule was demonstrated in [33]. This result is now also known to follow from the more general theory of PLS completeness for local optimization problems [87] Related work appears in [11, 30] Somewhat surprisingly, the very constrained convergence behavior of symmetric nets is not reflected in their computational power, at ....

Haken, A. Connectionist networks that need exponential time to stabilize. Unpublished manuscript, Dept. of Computer Science, University of Toronto, 1989.

....of Hop eld nets may indeed be exponential for both sequential [43] and parallel [40] modes. This is witnessed e.g. by symmetric, sequential or parallel implementations of a binary counter that traverses most of the network space before it converges in time i s=8 ) asynchronous sequential updates [42] or s=3 ) fully parallel steps [38, 39] For the average case, on the other hand, a very fast convergence O(log log s) of binary 11 Hop eld nets can be shown under reasonable assumptions [61] However, the previous bounds do not take into account the size of the weights. An upper bound of O(W ) ....

A. Haken, Connectionist networks that need exponential time to stabilize, Unpublished manuscript, Department of Computer Science, University of Toronto, 1989.

....and in [28] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [25] or [23] by a fairly simple construction. Finally, a network requiring exponential time for stabilization under any asynchronous update rule was demonstrated in [27]. Related work appears in [8, 24] Somewhat surprisingly, the very constrained convergence behavior of symmetric nets is not reflected in their computational power, at least not when the synchronous update rule is used. In this model, polynomial size symmetric nets with unbounded weights are ....

Haken, A. Connectionist networks that need exponential time to stabilize. Manuscript, 10 pp., January 1989.

....in [14] and in [19] for a particular sequential update rule. For a di erent sequential rule the result actually follows already from [17] or [14] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any sequential update rule was demonstrated in [18]. 2 3. FINITE BINARY STATE NETWORKS It has been known since the early work of McCulloch and Pitts [32] and Kleene [25] that nite binary state neural networks are equivalent to nite automata for processing sequentially given inputs. A somewhat interesting question here is how ecient are neural ....

.... output neurons (in the case of Boolean function computations, a single output neuron) To be able to process arbitrary input sizes, this formulation requires that the network grows with increasing input size, i.e. that we actually consider nonuniform sequences of networks, 2 The manuscript [18] remains unpublished, but the construction is reviewed in [37] The result can also be shown to follow, although in a somewhat convoluted way, from the more general theory of PLScompleteness for local optimization problems [44] one for each input size. If arbitrary changes to the network ....

[Article contains additional citation context not shown here]

Haken, A. Connectionist networks that need exponential time to stabilize. Manuscript, 10 pp., January 1989.

....require an exponential time to converge, as was first shown in [11] for synchronous updates (a simplified construction appears in [9] and in [13] for a particular asynchronous update rule. A network requiring exponential time to converge under any asynchronous update rule was demonstrated in [12]. Related work appears in [5, 10] In this paper, we prove that despite their apparently very constrained behavior also symmetric polynomial size networks are capable of computing all functions in PSPACE poly. The idea, presented in section 4, is to start with the simulation of space bounded ....

....work, asynchronous update rules are used. Are also asynchronous Hopfield nets capable of computing all of PSPACE poly Or, with the small weights restriction, all of P poly The synchronous Goles Mart inez counting network could possibly be replaced by an asynchronous counter due to A. Haken [12], but it is not clear how to effect the rest of the construction. 2. Also in Hopfield s original model, all the units are used for both input and output, and no hidden units are allowed. Although this is a somewhat artificial restriction from the function computation point of view, it would ....

Haken, A. Connectionist networks that need exponential time to converge. Manuscript, 10 pp., January 1989.

....require an exponential time to converge, as was first shown in [12] for synchronous updates (a simplified construction appears in [10] and in [14] for a particular asynchronous update rule. A network requiring exponential time to converge under any asynchronous update rule was demonstrated in [13]. Related work appears in [6, 11] In this paper, we prove that despite their apparently very constrained behavior also symmetric polynomial size networks are capable of computing all functions in PSPACE poly. The idea, presented in section 4, is to start with the simulation of space bounded ....

....network. One reasonable approach might be to require that for the function value to be defined at a given input point, all update orders must lead to the same result. The synchronous Goles Mart inez counting network could possibly be replaced by an asynchronous counter due to A. Haken [13], but it is not clear how to effect the rest of the construction. Also in Hopfield s original model, all the units are used for both input and output, and no hidden units are allowed. Although this is a somewhat artificial restriction from the function computation point of view, it would ....

Haken, A. Connectionist networks that need exponential time to converge. Manuscript, 10 pp., January 1989.

....time in the worst case. For more details on this approach, see Section 5.2. Here we present two explicit constructions: first a network due to Goles and Mart inez [29, 30] that requires an exponential number of fully parallel steps to converge, and then a more involved construction by Haken [32] of a network whose convergence time is exponential also in sequential operation 3 . Let us first consider fully parallel updates [29] For a given n, we construct a network of size linear in n that functions as an n bit binary counter, and thus takes time at least 2 n to converge. The ....

....4 nodes, and requires 2 n 2 n Gamma1 Gamma 3 update steps to converge, when started in the all 0 s state. For the exact details, see [29] or [30, pp. 88 95] Let us then turn to the more complicated case of sequential updates. The basic idea of the construction we present, due to Haken [32], is again to implement a binary counter, but the control of the network is now complicated by the necessity of preparing for all possible update orders. This time we work directly in the bipolar model. The basic unit of the design is the XOR subnetwork presented in Figure 4. It can be seen that ....

[Article contains additional citation context not shown here]

A. Haken. Connectionist Networks that Need Exponential Time to Stabilize. Unpublished manuscript, Department of Computer Science, University of Toronto, 1989.

....and in [34] for a particular asynchronous update rule. For a different update rule the latter result actually follows already from [31] or [29] by a fairly simple construction. Finally, a network requiring exponential time for convergence under any asynchronous update rule was demonstrated in [33]. This result is now also known to follow from the more general theory of PLS completeness for local optimization problems [87] Related work appears in [11, 30] Somewhat surprisingly, the very constrained convergence behavior of symmetric nets is not reflected in their computational power, at ....

Haken, A. Connectionist networks that need exponential time to stabilize. Unpublished manuscript, Dept. of Computer Science, University of Toronto, 1989.

....for synchronous updates, and by Haken and Luby (1988) for a particular asynchronous update rule. The former construction was later simplified by Goles and Mart inez (1989) A network requiring exponential time to converge under an arbitrary asynchronous update order was first demonstrated by Haken (1989). The existence of networks with exponentially long asynchronous transients is now known to follow also from the general theory of local search for optimization problems (Schaffer and Yannakakis 1991) In this paper, we prove that despite their constrained dynamics, computationally symmetric ....

Haken, A. 1989. Connectionist networks that need exponential time to converge. Unpublished manuscript, 10 pp. University of Toronto, Dept. of Computer Science, January 1989.

....the existence of symmetric networks with exponentially long transients under ordered sequential updates. Proving the existence of long transients under unordered sequential updates is quite a bit more complicated, however, and seems to have been worked out first by A. Haken in a manuscript [12] which, unfortunately, remains unpublished. On the other hand, the result is now known to follow, albeit via a somewhat indirect route, also from the general theory of local search for optimization problems [23] and the explicit construction of [12] is reviewed in [3] and also below. As ....

....worked out first by A. Haken in a manuscript [12] which, unfortunately, remains unpublished. On the other hand, the result is now known to follow, albeit via a somewhat indirect route, also from the general theory of local search for optimization problems [23] and the explicit construction of [12] is reviewed in [3] and also below. As another example, in [16] general scheme was presented for simulating polynomial space (resp. polynomial time) bounded Turing machines by symmetric polynomial size nets (resp. polynomial size nets with polynomially bounded connection weights) The ....

[Article contains additional citation context not shown here]

Haken, A. Connectionist networks that need exponential time to converge. Unpublished manuscript, Dept. of Computer Science, University of Toronto, January 1989. 10 pp.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC