A. Blum and R. Rivest. Training a 3-node neural network is NP-Complete. Neural Networks, 5:117--127, 1992.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....the complexity of the training problem should scale well with respect to the number of neurons. It turns out that the training problem is NP hard in several situations, i.e. the respective problems are infeasible (under the standard complexity theoretic assumption of P6=NP [14] Blum and Rivest [9] showed that a varying input dimension yields the NP hardness of the training problem for architectures with only two hidden neurons using the threshold activation functions. The approaches in [15, 27] generalize this result to multilayered threshold networks. Investigation has been done to get ....

....1= 68n 1 2 1 170n 1 ) and multilayer threshold networks with varying input dimension and a fixed number n 1 of neurons in the first hidden layer for any n 1 2. In Section 3.2. 1, we show that, for architectures with one hidden layer and two hidden neurons (the classical case considered by [9]) approximation of mL with relative error smaller than 1=c is NP hard even if either (a) c = 2244, the threshold activation function in the hidden layer is substituted by the classical sigmoid function, and the situation of separation in the output is considered, or (b) c = 2380 and the ....

[Article contains additional citation context not shown here]

A. Blum and R. L. Rivest, Training a 3-node neural network is NP-complete, Neural Networks 5, pp. 117-127, 1992.

....by White [61] shows how allowing a network to grow as it learns will give it the advantage of being able to essentially learn any arbitrary mapping. In fact, the addition of nodes solves the issue that the solution of weights in a fixed neural network architectures is a NP complete problem [62]. Baum [63] observed that structure adding networks are complete representations, i.e. these networks can learn any problem in polynomial time. In the context of CALM, these considerations are only partly applicable, since we limit ourselves to only grow or shrink individual modules. In other ....

A. Blum & R. L. Rivest, Training a 3-node neural network is NP-complete, In: D. S. Touretzky (Ed.), Advances in neural information processing systems, Morgan Kaufmann, 1988.

....the complexity of the training problem should scale well with respect to the number of neurons. It turns out that the training problem is NP hard in several situations, i.e. the respective problems are infeasible (under the standard complexity theoretic assumption of P6=NP [15] Blum and Rivest [9] showed that a varying input dimension yields the NP hardness of the training problem for architectures with only two hidden neurons using the threshold activation functions. The approaches in [16,28] generalize this result to multilayered threshold networks. Investigation has been done to get ....

....1= 68n 1 2 1 170n 1 ) and multilayer threshold networks with varying input dimension and a fixed number n 1 of neurons in the first hidden layer for any n 1 2. In Section 3.2. 1, we show that, for architectures with one hidden layer and two hidden neurons (the classical case considered by [9]) approximation of m L with relative error smaller than 1=c is NPhard even if either (a) c = 2244, the threshold activation function in the hidden layer is substituted by the classical sigmoid function, and the situation of separation in the output is considered, or (b) c = 2380 and the ....

[Article contains additional citation context not shown here]

A. Blum and R. L. Rivest, Training a 3-node neural network is NP-complete, Neural Networks 5 (1992) 117--127.

....FNN with the Heavyside activation function can be done in polynomial time. However, most existing algorithms do not take the specific architecture into account. Hence the loading problem should be considered in a more general setting taking arbitrary architectures as input. Starting with the work [20, 75], it is known that general neural network training is NP hard. More precisely, the paper of Blum and Rivest [20] states the fact that the loading problem is NP hard for multilayer architectures with the Heavyside activation function even if the number of hidden neurons is fixed to two and only the ....

....take the specific architecture into account. Hence the loading problem should be considered in a more general setting taking arbitrary architectures as input. Starting with the work [20, 75] it is known that general neural network training is NP hard. More precisely, the paper of Blum and Rivest [20] states the fact that the loading problem is NP hard for multilayer architectures with the Heavyside activation function even if the number of hidden neurons is fixed to two and only the number of inputs is allowed to vary from one instance to the next one. Training a single perceptron is, of ....

A. Blum and R. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 9:1017 1023, 1988.

....optima. For instance inductive decision trees employ a greedy local optimization approach, and neural networks apply gradient descent techniques to minimize an error function over the training data. Moreover optimal training with finite data both for neural networks and decision trees is NP hard [13, 57]. As a consequence even if the learning algorithm can in principle find the best hypothesis, we actually may not be able to find it. Building an ensemble using, for instance, di#erent starting points may achieve a better approximation, even if no assurance of this is given. Another way to look at ....

A. Blum and R.L. Rivest. Training a 3-node neural network is NP-complete. In Proc. of the 1988.

....that the problem of building the optimal linear decision trees is NP hard. However, one might hope that, by reducing the size of the decision tree, or the dimensionality of the data, it might be possible to make the problem tractable This does not seem to be the case either Blum and Rivest [32] showed that the problem of constructing an optimal 3 node neural network is NP complete. Goodrich [176] proved that optimal (smallest) linear decision tree construction is NP complete even in three dimensions 2.6.2 Other analytical results Goodman and Smyth [174] showed that greedy top down ....

....a small tree. The problem of inducing the smallest axis parallel decision tree is known to be NP hard (Section 2.6.1) It is also easy to see that the problem of constructing an optimal (e.g. smallest) oblique decision tree is NP hard. This conclusion follows from the work of Blum and Rivest [32]. Their result implies that in d dimensions (i.e. with d attributes) the problem of producing a 3 node oblique decision tree that is consistent with the training set is NP complete. More specifically, they show that the following decision problem is NP complete: given a training set T with n ....

A. BLUM AND R. RIVEST. Training a 3-node neural network is NP-complete. In Proceedings of the 1988.

....weight space due to spurious minima of the error function even for very simple architectures [SS2] Second, learning is at least as complex as in the case of feedforward networks. Here the loading problem, which is a decision problem correlated to the learning task, is NP hard for some situations [BR, DSS, H2]. Third, if no further information about the probability of the examples used for training is available, an information theoretical barrier can prevent a guarantee for valid generalization. This is due to the infiniteness of the Vapnik Chervonenkis (VC) dimension for standard function classes. In ....

A. Blum and R. Rivest, Training a 3-node neural network is NPcomplete, Neural Networks, 5 (1992), 117-127.

....) If n 1 3, then the above result is true with = c(n 1 1) 5n 1 ) for some positive constant c, even if restricted to situations where a solution without any misclassi cations exists. Restricted to architectures with only one hidden layer and two hidden neurons (as in, for example, [6]) approximation of r with relative error at most 1=c is NP hard even if either (a) c = 2244, the threshold activation function in the hidden layer is substituted by the classical sigmoid function, and the situation of separation of the output classi cation is assumed, or (b) c = 2380 and ....

.... for polynomial separability [8, 10, 19] For some strange activation functions or a setting where the number of examples is appropriately restricted with respect to the number of hidden neurons loadability is trivial in the sence that every pattern set can be loaded [23] However, Blum and Rivest [6] show that a varying input dimension yields the NP hardness of training neural nets for the threshold activation function and an architecture with only two hidden neurons. Hammer [10] states a generalization of this result to arbitrary multilayered networks with the threshold activation function. ....

[Article contains additional citation context not shown here]

A. Blum and R. L. Rivest, Training a 3-node neural network is NP-complete, Neural Networks 5, pp. 117-127, 1992.

....to consider the principle complexity of the learning task. Here, we will deal with an even simpler correlated problem, just to decide if a pattern set can be stored correctly with a neural network. This is NP complete for a 3 node network with perceptron activation with varying input dimension [1] and some results for the sigmoidal activation exist as well [2, 3] But the results are restricted to the 3 node resp. other special architectures. Here, the completeness result in the perceptron case is generalized to an arbitrary multilayer feedforward network with at least 2 nodes in the first ....

....i.e. varying (n; h; n 1 ; n h ) the problem is still in NP, because we can guess appropriate weights they can be chosen polynomial in the input set and test if the network maps the patterns correctly. But the problem becomes NP complete even if h = 1, n 1 = 2 and only n varies [1]. Of course this could go back to the very restricted function class with only two hidden neurons. Indeed, if we take n 1 m the problem becomes tractable because in this case any consistent pattern set can be loaded. Therefore the question arises whether loading is intractable for an ....

A. Blum and R. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 5:117--127, 1992.

....sometimes very slow, especially for large input dimensions. The loading problem is to decide if a training set can be stored by a fixed architecture correctly. Blum and Rivest have shown the NP completeness for a network with 3 computation units, varying input dimension, and perceptron activation [1]. Dasgupta et.al. have generalized the result to the semilinear activation [2] Usually, one deals with the sigmoidals sgd(x) or tanh(x) Hoeffgen has proved the NP completeness for sgd, but binary weights [4] S ima has shown the NP hardness for a sigmoidal architecture with an additional ....

A. Blum and R. Rivest. Training a 3-node neural network is NP-complete. In First Workshop on Comp. Learning Theory. Morgan-Kaufmann, 1988.

....layer threshold circuits with n hidden nodes and varying input dimension, approximation of this ratio within a relative error c=n , for some positive constant c, is NP hard even if the number of examples is limited with respect to n. For architectures with two hidden nodes (e.g. as in [6]) approximating the objective within some fixed factor is NP hard even if any sigmoid like activation function in the hidden layer and separation of the output [19] is considered, or if the semilinear activation function substitutes the threshold function. Next, we consider the objective to ....

.... function or architectures with appropriately restricted connection graph loading is polynomial [8, 10, 15, 20] For some strange activation functions or a setting where the number of examples coincides with the number of hidden nodes loadability becomes trivial [25] However, Blum and Rivest [6] show that a varying input dimension yields the NP hardness of training threshold circuits with only two hidden nodes. Hammer [10] generalizes this result to multilayered threshold circuits. References [8, 11, 12, 14, 23, 27] constitute generalizations to circuits with the sigmoidal activation ....

[Article contains additional citation context not shown here]

A. Blum and R. L. Rivest, Training a 3-node neural network is NP-complete, Neural Networks 5, pp. 117-127, 1992.

....problems to consistency problems suffer a little bit from using quite artificial architectures which are not likely to be used in any existing learning environment. Nevertheless, his results are among the first rigorously proven negative results concerning neural learning. Blum and Rivest in [6] showed nonlearnabilty for quite simple architectures: one hidden layer with k 2 hidden units and one output unit which computes the logical AND. This architecture represents polyhedrons with k facets, i.e. intersections of k halfspaces. Our paper followed this tendency of simplification by ....

A. Blum and R. L. Rivest, Training a 3-node neural network is NPcomplete, in "Proceedings of the 1988 Workshop on Computational Learning Theory," pp. 9--18.

....A SURVEY 12 2.1. 3 Computational motivations Complexity of learning An difficult NN performance problem is the choice of the set of neural network weights which performs the desired mapping or classification function (sometimes called the loading problem) Although this problem is NP complete [BR88] there are greedy algorithms (e.g. gradient search algorithms) which can offer good approximations [RHW86, HH93] However, such algorithms suffer from the high coupling (interdependency) they introduce in training different hidden nodes [JMM90, AKR94] This high coupling does not cope with the ....

A. Blum and R. Rivest. Training a 3-node neural network is np-complete. In The first ACM Workshop on the Computatioal Learning Theory, pages 9--18, Cambridge, MA, USA, Aug. 1988.

....hard to learn in the PAC model using any hypotheses. However, such classes are too rich to be considered useful for learning purposes. Pitt and Valiant [22] showed that it is NP hard to decide if there is a 2 term DNF that correctly classifies all examples in a training sample. Blum and Rivest [6] established a similar result for two layer linear threshold networks with only two hidden units, and DasGupta, Siegelmann and Sontag [8] extended these results to apply to networks with piecewise linear hidden units. The earliest hardness results for agnostically learning simple classes ....

A.L. Blum and R.L. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 5(1):117--127, 1992.

No context found.

A. Blum and R. Rivest. Training a 3-node neural network is NP-Complete. Neural Networks, 5:117--127, 1992.

No context found.

Blum, A., & Rivest, R. L. (1988). Training a 3-node neural network is NP-complete. Proceedings of the 1988 IEEE Conference on Neural Information (pp. 494-501). Morgan Kaufmann.

No context found.

Blum, A. L., & Rivest, R. L. (1992). Training a 3-node neural network is NPcomplete. Neural Networks, 5 (1), 117--127.

No context found.

Blum, A. L., & Rivest, R. L. (1990). Training a 3-node neural network is NPcomplete. Tech. Rep., MIT, Cambridge, MA.

No context found.

A.L. Blum and R.L. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 5(1):117--127, 1992.

No context found.

A. Blum & R. L. Rivest, Training a 3-node neural network is NP-complete, In: D. S. Touretzky (Ed.), Advances in neural information processing systems, Morgan Kaufmann, 1988.

No context found.

A.L. Blum and R.L. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 5:117--127, 1992. 45

No context found.

A. Blum and R. Rivest. Training a 3-node neural network is NP-complete. Neural Networks, 9:1017--1023, 1988. 196

No context found.

Blum, A., and Rivest, R. L. Training a 3-node neural network is NP-complete. In Proceedings of the

No context found.

Blum, A. and Rivest, R. L. Training a 3-node neural network is np-complete. In Proceedengs of the First ADM Workshop on the Computational Learning Theory, pp. 9-18, Cambrige, MA, 1988.

No context found.

A. Blum and R. L. Rivest. Training a 3-node Neural Network is NP-complete. Neural Networks, 5(1), 1992: 117--127.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC