J.L. Balcazar, R. Gavalda and H.T. Siegelmann, Computational power of neural networks: A characterization in terms of Kolmogorov complexity, IEEE Transactions of Information Theory 43 (4) (1997) 1175-1183.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....Universality of Finite Analog Hopfield Nets In this section we will deal with the computational power of finite analog state discrete time recurrent neural networks. For the asymmetric analog networks, the computational power is known to increase with the Kolmogorov complexity of real weights [2]. With integer weights such networks are equivalent to finite automata [14, 15, 30] while with rational weights arbitrary Turing machines can be simulated [15, 25] With arbitrary real weights the network can even have super Turing computational capabilities, e.g. polynomial time computations ....

Balc'azar, J. L., Gavald`a, R., Siegelmann, H. T. Computational power of neural networks: A characterization in terms of Kolmogorov complexity. IEEE Transactions of Information Theory, 43, 1175--1183, 1997.

....exponential time any input output mapping can be computed. Moreover, a proper hierarchy of nonuniform complexity classes between P and P poly has been proved for polynomial time computations of analog recurrent networks with increasing Kolmogorov complexity (information contents) of real weights [8]. For example, setting a logarithmic bound on the resource bounded Kolmogorov complexity of the real weights, the languages recognized correspond to the complexity class Full P log (see [9] for a de nition) All the preceding results concerning analog computations in nite recurrent networks ....

J.L. Balcazar, R. Gavalda, and H.T. Siegelmann, Computational power of neural networks: A characterization in terms of Kolmogorov complexity, IEEE Transactions of Information Theory 43 (4) (1997) 1175-1183.

....exponential time any input output mapping can be computed. Moreover, a proper hierarchy of nonuniform complexity classes between P and P poly has been proved for polynomial time computations of analog recurrent networks with increasing Kolmogorov complexity (information contents) of real weights [8]. For example, setting a logarithmic bound on the resource bounded Kolmogorov complexity of the real weights, the languages recognized correspond to the complexity class Full P log (see [9] for de nition) All the preceding results concerning analog computations of nite recurrent networks assume ....

J.L. Balcazar, R. Gavalda, and H.T. Siegelmann, Computational power of neural networks: A characterization in terms of Kolmogorov complexity, IEEE Transactions of Information Theory 43 (4) (1997) 1175-1183.

....perceptrons [9] The computational properties of traditional perceptron networks seem to be well understood now. The computational power of the nite recurrent discretetime neural networks with saturated linear activation function increases with the Kolmogorov complexity of real weight parameters [2]. For example, such networks with integer weights (corresponding to binary state networks with threshold units) are equivalent to nite automata [14] while those with rational weights can simulate arbitrary Turing machines [12, 24] In addition, ner descriptive measures were introduced for the ....

Balcazar, J. L., Gavalda, R., Siegelmann, H. T. Computational power of neural networks: A characterization in terms of Kolmogorov complexity. IEEE Transactions of Information Theory, 43, 1175-1183, 1997.

....perceptrons [9] The computational properties of traditional perceptron networks seem to be well understood now. The computational power of the nite recurrent discretetime neural networks with saturated linear activation function increases with the Kolmogorov complexity of real weight parameters [2]. For example, such networks with integer weights (corresponding to binary state networks with threshold units) are equivalent to nite automata [14] while those with rational weights can simulate arbitrary Turing machines [12, 24] In addition, ner descriptive measures were introduced for the ....

Balcazar, J. L., Gavalda, R., Siegelmann, H. T. Computational power of neural networks: A characterization in terms of Kolmogorov complexity. IEEE Transactions of Information Theory, 43, 1175-1183, 1997.

.... In this section we deal with the computational power of nite analog state discrete time recurrent neural networks with the saturated linear activation function (3) For asymmetric analog networks, the computational power is known to increase with the Kolmogorov complexity of their real weights (Balc azar et al. 1997). With integer weights such networks are equivalent to nite automata (Horne and Hush, 1996; Indyk, 1995; S ma and Wiedermann, 1998) while with rational weights arbitrary Turing machines can be simulated (Indyk, 1995; Siegelmann and Sontag, 1995) With arbitrary real weights such networks ....

Balcazar, J. L., Gavalda, R., and Siegelmann, H. T. (1997). Computational power of neural networks: A characterization in terms of Kolmogorov complexity. IEEE Transactions of Information Theory, 43:1175-1183.

....between two subsequent discrete updates corresponds to a continuous time unit. This result suggests that continuous time analog models of computation may be worth investigating more for their eciency gains than for their (theoretical) capability for arbitrary precision real number computation [2, 15, 16]. The predecessors of the present work are: a similar, but considerably simpler, construction used in [12] to prove the computational equivalence ofsymmetric and convergent asymmetric discrete time binary networks 1 , and the simulation of discrete time networks by asymmetric continuous time ....

....A related line of study concerns the computational power of nite discrete time analog state neural networks. Here it is known that the computational power of asymmetric networks using the saturatedlinear sigmoid activation function increases with the Kolmogorov complexity of the weight parameters [2]. With integer weights such networks are equivalent to nite automata [8, 9, 18] while with rational weights arbitrary Turing machines can be simulated [9, 16] With arbitrary real weights the networks can even have super Turing computational capabilities [15] On the other hand, it is known ....

Balcazar, J. L., Gavalda, R., Siegelmann, H. T. Computational power of neural networks: A characterization in terms of Kolmogorov complexity. In IEEE Transactions of Information Theory, 43, 1175-1183, 1997.

.... In this section we deal with the computational power of nite analog state discretetime recurrent neural networks with the saturated linear activation function (3) For asymmetric analog networks, the computational power is known to increase with the Kolmogorov complexity of their real weights [4]. With integer weights such networks are equivalent to nite automata [23, 24, 47] while with rational weights arbitrary Turing machines can be simulated [24, 41] With arbitrary real weights such networks can even have super Turing computational capabilities, so that e.g. polynomial time ....

J. L. Balcazar, R. Gavalda, and H. T. Siegelmann. Computational power of neural networks: A characterization in terms of Kolmogorov complexity. IEEE Transactions of Information Theory, 43:1175-1183, 1997.

.... 1 0 for 0 : 4.1) Hence, the states of analog neurons are real numbers within the interval [0; 1] and similarly the weights (including biases) are allowed to be reals. The computational power of asymmetric analog networks is known to increase with the Kolmogorov complexity of real weights [2]. With integer weights such networks are equivalent to finite automata [14, 15, 28] while with rational weights arbitrary Turing machines can be simulated [15, 24] With arbitrary real weights the network can even have super Turing computational capabilities, e.g. polynomial time computations ....

Balc'azar, J. L., Gavald`a, R., Siegelmann, H. T. Computational power of neural networks: A characterization in terms of Kolmogorov complexity. IEEE Transactions of Information Theory, 43, 1175--1183, 1997.

....related line of study concerns the computational power of finite discrete time analog state neural networks. Here it is known that the computational power of asymmetric networks using the saturated linear sigmoid activation function increases with the Kolmogorov complexity of the weight parameters [2]. 4 On the other hand, it is known that any amount of analog noise reduces the computational power of this model to that of finite automata [9] In the present abstract we outline our proof construction, and give a simulation example witnessing its validity. The formal verification of the ....

Balc'azar, J. L., Gavald`a, R., Siegelmann, H. T. Computational power of neural networks: A characterization in terms of Kolmogorov complexity. In IEEE Transactions of Information Theory, 43, 1175--1183, 1997.

....related line of study concerns the computational power of finite discrete time analog state neural networks. Here it is known that the computational power of asymmetric networks using the saturatedlinear sigmoid activation function increases with the Kolmogorov complexity of the weight parameters [2]. 2 On the other hand, it is known that any amount of analog noise reduces the computational power of this model to that of finite automata [9] In the present abstract we outline our proof construction, and give a simulation example witnessing its validity. The formal verification of the ....

Balc'azar, J. L., Gavald`a, R., Siegelmann, H. T. Computational power of neural networks: A characterization in terms of Kolmogorov complexity. In IEEE Transactions of Information Theory, 43, 1175--1183, 1997.

....All algebraic numbers, constants such as and e, and many others are polynomial time computable. To emphasize how small this class is, we note that there are no more polynomial time computable real numbers than Turing machines, hence there are countably many of them. Furthermore it can be shown [3] that, when used as constants in ARNN, networks still compute the class P only, just like in the case where all constants are rational numbers. As the first evidence of the arithmetic networks superiority, we prove that arithmetic networks can recognize some recursive functions arbitrarily faster ....

....computable weights, and yet they accept recursive languages of arbitrarily high time complexity (in the Turing machine sense) Again, this is in contrast with the first order case and the rational weight case. Firstorder nets with polynomial time computable weights accept only languages in P [3], and arithmetic nets with rational weights can be simulated in PSPACE, so in exponential time. Theorems 3.1 and 3.2 are both consequences of the following theorem. Theorem 3.3 For every time constructible function t(n) there is a net N in (IRpoly, foe; oe H g) such that: 1. The weights in N are ....

J.L. Balc'azar, R. Gavald`a, and H.T. Siegelmann, "Computational power of neural networks: A characterization in terms of Kolmogorov complexity". IEEE Transactions on Information Theory 43 (1997), 1175--1183.

No context found.

J.L. Balcazar, R. Gavalda and H.T. Siegelmann, Computational power of neural networks: A characterization in terms of Kolmogorov complexity, IEEE Transactions of Information Theory 43 (4) (1997) 1175-1183.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC