A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In David S. Touretzky, editor, Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, San Mateo, CA, 1989.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....(pairs (x i ; y i ) whether or not there is some f 2 F that is consistent on S, i.e. such that for each i, f(x i ) y i ) This was used to show that if the consistency problem for F is NP hard, then F is not PAC learnable unless RP = NP. The technique has seen a number of applications [51, 15, 26, 14]. Because learning with equivalence queries implies PAC learning, it follows that for such classes F , F cannot be learned with equivalence queries unless RP = NP. A folklore extension of this observation, which we present in Section 4 for completeness, directly relates the consistency problem to ....

....[16] and in the equivalence query model [47] in both cases the algorithms work without membership queries. When k 2, e.g. for learning the union of two halfspaces (or neural nets with two hidden nodes and one root level OR ) Blum and Rivest showed that the consistency problem is NP hard [14], and thus this class of functions cannot be learned in polynomial time in the PAC model if RP 6= NP, nor in the equivalence query model if P 6= NP. Whether or not the union of two halfspaces can be properly learned (i.e. using hypotheses that are unions of two halfspaces) in polynomial time ....

[Article contains additional citation context not shown here]

A. Blum and R. L. Rivest, Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494--501, Morgan Kaufmann, Sam Mateo, CA, 1989.

....a value of m L ( sucient for pac learning that is polynomial in n and 1= Judd [15] was the rst to show that learning in neural networks can be hard, in the formal complexity theoretic sense. We now describe a simple hardness result from [3,4] along the lines of one due to Blum and Rivest [7]. Before doing so, we recall that, in complexity theory, two important classes of problems, rp and np, are de ned. The class rp is the class of all problems that can be solved by randomized algorithms in polynomial time, while np is the class of problems that can be solved by non deterministic ....

....there is a polynomial time algorithm for N k consistency, then there is one for graph k colouring. But graph k colouring is np complete for k 3, and it follows that the N k consistency problem is np hard if k 3. In fact, the same is true if k = 2: this follows from work of Blum and Rivest [7]. Theorem 6.3 enables us to move from this hardness result for the consistency problem to a hardness result for pac learning. The theorem tells us that, unless rp=np, there can be no computationally ecient pac learning algorithm for this family of neural networks. ....

A. Blum and R. L. Rivest, Training a 3-node neural net is NP-Complete, in Advances in Neural Information Processing Systems I, (ed. D. S. Touretzky), Morgan Kaufmann, 1989, pp. 494-501.

....for H which runs in time polynomial in ffl Gamma1 and n. ut The fact that computational complexity theoretic hardness results hold for neural networks was first shown by Judd [61] In this section we shall prove a simple hardness result from [9, 10] along the lines of one due to Blum and Rivest [34]. The network has n inputs and k 1 computation units (k 1) The first k computation units are in parallel and each of them is connected to all the inputs. The last computation unit is the output unit; it is connected by arcs with fixed weight 1 to the other computation units, and it has fixed ....

....polynomial time algorithm for N k GammaCONSISTENCY, then there is one for GRAPH k COLORING. But GRAPH k COLORING is NP complete [44] and hence it follows that the N k GammaCONSISTENCY problem is NP hard if k 3. In fact, the same is true if k = 2. This follows from work of Blum and Rivest [34]. Thus, fixing k, we have a very simple family of feedforward linear threshold networks, each consisting of k 1 computation units (one of which is hardwired and acts simply as an AND gate) for which the problem of loading a training sample is computationally intractable. Theorem 8.2 ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In D. S. Touretzky, editor, Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

....units for exploring the scaling up behavior. In practice, however, we usually do choose the most appropriate architecture for solving a given task, and therefore, we deal with a problem which is signi cantly di erent with respect to that investigated by Judd (loading problem) 3] Blum and Rivest [6] have studied the computational complexity of the problem of learning in a net with a xed number of neurons when increasing the number of inputs and the number of patterns. Again, as pointed out by Baum [3] the negative result which is found, that states that the problem is NP complete, does ....

A. Blum and R. Rivest, \Training in 3-node neural net is NP-complete", in: D.S. Touresky Ed. Advance in Neural Information Processing Systems I, San Mateo, CA: Morgan Kaufman, 1989, pp. 494-501.

....alternations in label. Figure 1 shows an example of a function with 6 alternations. This is a rare case in which it is possible to nd a global minimum of the training error in a reasonable amount of time, as opposed to many learning problems in which nding such minimum is an intractable problem [7, 4, 2, 1]. This makes this problem ideal for analyzing the behavior of our model selection algorithm, since the computation of the Rademacher penalization term requires nding a minimum of the training error with respect to relabeled data and we do not want our results to be obscured by the presence of ....

A. Blum and R. L. Rivest. Training a 3-node neural net is np-complete. In David S. Touretsky, editor, Advances in Neural Information Processing Systems I, pages 494-501, San Mateo, CA, 1989. Morgan Kaufmann.

.... w Delta x : A function g is an intersection of two halfspaces if there are halfspaces h 1 ; h 2 such that g(x) 1 iff h 1 (x) h 2 (x) 1 (i.e. g = h 1 h 2 ) The concept class of intersections of halfspaces is a natural one which has been widely studied in computational learning theory [2, 3, 4, 5, 6, 14, 15, 20]. We consider the following problem: SPECIFICATION NUMBER FOR INTERSECTION OF HALFSPACES (SNIH) Instance: Two halfspaces h 1 and h 2 over f0; 1g d (each of which is represented by a (d 1) tuple (w 1 ; w d ; 2 R d 1 ) and an integer k 2 d : Question: Is jSj k; where S is an ....

A. Blum and R. Rivest. Training a 3-node neural net is NP-complete, in D. Touretzky, ed., "Advances in Neural Information Processing Systems I," 1989, 494-501.

....as a possibility for m i fails to hold for some positive example (i.e. because s i = 1 in this case) then it is eliminated. We note that in addition to a substantial literature on positive results in this area, a theory of some of the ultimate limitations to inductive learning also exists [BR92, PV88, EHKV89, KV94a]. While the majority of this work is restricted to the case in which in all examples all the attribute values are defined (i.e. there is no equivalent to our ) the more general case has received some discussion also [SG94, KR94b, Val84] b) Logical deduction: As previously noted logical ....

A. Blum and R.L. Rivest. Training a 3-node neural net is NP-complete. Neural Networks, 5:117--127, 1992.

....instead of bounding hyperplanes, of the polytope. It follows that if one way functions exist, then learning convex polytopes is intractable. Learning convex polytopes in arbitrary dimension remains difficult even if the number of halfspaces is restricted to some small constant. Blum and Rivest [BR89] showed that finding an intersection of two halfspaces that is consistent with a sample of labeled points from the boolean domain, if it exists, is NP complete. It has been shown that in the boolean domain, exact learning from equivalence and membership queries remains NPhard if the algorithm is ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494-- 501. Morgan Kaufmann, 1989.

....[16, 17] For poorly constrained architectures, such as threshold units, Judd shows that deciding if a given task is learnable by a given network architecture can take exponential time. The case of multilayer perceptrons, with threshold units, has been addressed by Blum and Rivest also. In [4], they prove that the learning problem is NP hard even with 3 hidden units. More recently, Vu has shown that learning an arbitrary number of data is NP hard even if exact learning is not required [25] These theoretic hardness results of computational complexity only concern learning machines ....

A. Blum and R. Rivest. Training a 3-node neural net is NP-complete. In Advances in Neural Information Processing Systems I, pages 494--501, 1989.

....queries in poly(log m) time. Recently, Blum et al. 14] learn single half space in the presense of random classification noise using a simple greedy method to find a weak hypothesis and then apply a boosting technique to achieve the desired accuracy. Learning two halfspaces: Blum and Rivest [15] show that finding an intersection of two halfspaces that are consistent with a sample of labeled points from the boolean domain, if it exists, is NP Complete. Baum [16] presents an algorithm that learns intersections of two halfspaces from examples and membership queries, or from examples alone ....

....than the sample size there must be some amount of compression of the sample data. Polly will actually produce no more hyperplanes than the number in the target. This is the task found NP hard when the domain is restricted to the boolean hypercube in the PAC model without membership queries [15], and in the more demanding exact learning model with both membership and equivalence queries [19, 20] Let S and S Gamma denote the positive and negative examples of S, respectively. To understand the method that Polly uses to find a collection of at most s hyperplanes whose intersection ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

....to find a function that minimizes this objective function. In virtually all interesting cases, the computational problem of minimizing the objective function is NP hard. For example, fitting the weights of a neural network or finding the smallest decision tree are both NP complete problems [1, 4]. Hence, heuristic algorithms such as gradient descent (for neural networks) and greedy search (for decision trees) have been applied with great success. Of course, the sub optimality of such heuristic algorithms immediately suggests a reasonable line of research: find algorithms that can search ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

....queries in poly(log m) time. Recently, Blum et al. BFKV96] learn single half space in the presense of random classification noise using simple greedy method to find weak hypothesis and then apply boasting technique to achieve the desired accuracy. Learning two halfspaces: Blum and Rivest [BR89] show that finding an intersection of two halfspaces that are consistent with a sample of labeled points from the boolean domain, if it exists, is NP Complete. Baum [Bau90a] presents an algorithm that learns intersections of two halfspaces from examples and membership queries, or from examples ....

....than the sample size there must be some amount of compression of the sample data. Polly will actually produce no more hyperplanes than the number in the target. This is the task found NP hard when the domain is restricted to the boolean hypercube in the PAC model without membership queries [BR89] and in the more demanding exact learning model with both membership and equivalence queries [AHHP96, PR94] Let S and S Gamma denote the positive and negative examples of S, respectively. To understand the method that Polly uses to find a collection of at most s hyperplanes whose ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

....these linear discriminant functions do is either to reduce the complexity of the original problem or to transfer the original problem into two smaller problems. As theoretically and empirically shown previously, the training time of an MLP often increases exponentially with the size of the problem [2, 32, 35]. Thus, a real world problem (e.g. image processing) will be often intractable when the MLP is directly used. In our growing algorithm, linear discriminant functions first partition a large problem into several smaller problems prior to training of MLPs or subnetworks in order to limit the scale ....

A. Blum and R. Rivest. Training a 3-node neural net is NP-complete. In D. S. Touretsky, editor, Advances in Neural Information Processing Systems, pages 494--501, San Mateo, CA, 1989. Morgan Kaufmann.

....this class must have W(sd log n) parallel time which is not efficient (i.e. it is not poly logarithmic) 3 Previous Work Considerable work has been done on learning geometric concepts in the PAC model. In particular, unions and intersections of halfspaces have been considered. Blum and Rivest [8] show that there does not exist an efficient proper 4 learning algorithm for unions of s halfspaces, unless P = NP . Baum [5] gives an algorithm that efficiently learns a union of s halfspaces in a constant number of dimensions. 4 A learning algorithm is proper if all hypotheses come from the ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

....for this problem has a ratio bound of ln jU j 1 on the size of the approximation. 4 Previous Work In this section we highlight some of the relevant learning results for geometric concepts. There have been many results for the classes of unions and intersections of halfspaces. Blum and Rivest [11] show that there does not exist an efficient proper 4 learning algorithm for unions of s halfspaces, unless RP = NP . They also give an algorithm to PAC learn the xor of two halfspaces by transforming the examples so that positive and negative points are linearly separable. Baum [7] gives an ....

Avrim L. Blum and Ronald L. Rivest. Training a 3-node neural net is NP-Complete. Neural Networks, 5(1):117-- 127, 1992.

.... decision tree that can be used as a discriminator for a given set to multi category points in IR 3 (indeed, we define the problem for two categories: red and blue ) It is well known, for example, that constructing a best decision tree in general settings [22, 23] or in arbitrary dimensions [7, 26] is NP complete, but in the context of fixed dimensional decision tree approximations, however, each of these NP completeness proofs fail. Indeed, each of their respective optimization problems are polynomial time solvable in a fixeddimensional setting. Thus, some may have been tempted to believe ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. Neural Networks, 5:117--127, 1992.

....for H which runs in time polynomial in ffl Gamma1 and n. The fact that computational complexity theoretic hardness results hold for neural networks was first shown by Judd [66] In this section we shall prove a simple hardness result from [10, 11] along the lines of one due to Blum and Rivest [36]. Neural Computing Surveys 1, 1 47, 1997, http: www.icsi.berkeley.edu jagota NCS 23 The network has n inputs and k 1 computation units (k 1) The first k computation units are in parallel and each of them is connected to all the inputs. The last computation unit is the output unit; it is ....

....polynomial time algorithm for N k GammaCONSISTENCY, then there is one for GRAPH k COLORING. But GRAPH k COLORING is NP complete [47] and hence it follows that the N k GammaCONSISTENCY problem is NP hard if k 3. In fact, the same is true if k = 2. This follows from work of Blum and Rivest [36]. Thus, fixing k, we have a very simple family of feedforward linear threshold networks, each consisting of k 1 computation units (one of which is hard wired and acts simply as an AND gate) for which the Neural Computing Surveys 1, 1 47, 1997, http: www.icsi.berkeley.edu jagota NCS 24 ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In D. S. Touretzky, editor, Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

....decision tree. 1.1 Being Global is Hard One of the main challenges in using a decision tree for the learning problem is in quickly constructing a tree that is accurate. It is well known, for example, that constructing a best decision tree in general settings [14, 15] or in arbitrary dimensions [7, 17] is NP complete; hence, it is extremely unlikely that we will ever discover a polynomial time algorithm for constructing a best decision tree in these contexts. In the context of fixed dimensional learning, however, each of these NP completeness proofs fail. Indeed, each of their respective ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. Neural Networks, 5:117--127, 1992.

No context found.

Blum, A. and Rivest, R. L. (1989) Training a 3-node neural net is NP-Complete. In David S. Touretzky, editor, Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann.

....class of intersections of two homogeneous halfspaces in the standard PAC with queries model 3 . The idea of Baum s algorithm is to reduce the problem of learning an intersection of two homogeneous halfspaces to the problem of learning an XOR of halfspaces, for which a PAC algorithm exists [9]. That algorithm works by noticing that the XOR of v Delta x 0 and w Delta x 0 is equivalent to the degree 2 threshold function ( v Delta x) w Delta x) 0 (except on degenerate inputs) which can be learned as a linear threshold function over an O(n 2 ) dimensional space. The ....

....of the negative examples x 2 S which have the property that a membership query to Gamma x returns positive . Then find a linear function P such that P ( x) 0 for all the marked (negative) examples and P ( x) 0 for all the positives. Finally, run the XOR of halfspaces learning algorithm of [9] to find a hypothesis H 0 that correctly classifies f x 2 S : P ( x) 0g. The final hypothesis is: If P ( x) 0 then predict negative, else predict H 0 ( x) Baum s algorithm seems appropriate for our model because it does not explicitly try to query examples near the boundary. In fact, it ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

....class of intersections of two homogeneous halfspaces in the standard PAC with queries model 2 . The idea of Baum s algorithm is to reduce the problem of learning an intersection of two homogeneous halfspaces to the problem of learning an XOR of halfspaces, for which a PAC algorithm exists [7]. That algorithm produces a hypothesis that is the threshold of a degree 2 polynomial. The idea of the reduction is to notice that negative examples in the quadrant opposite from the positive quadrant the troublesome examples keeping the data set from being consistent with an XOR of ....

....of the negative examples x 2 S which have the property that a membership query to Gamma x returns positive . Then find a linear function P such that P ( x) 0 for all the marked (negative) examples and P ( x) 0 for all the positives. Finally, run the XOR of halfspaces learning algorithm of [7] to find a hypothesis H 0 that correctly classifies f x 2 S : P ( x) 0g. The final hypothesis is: If P ( x) 0 then predict negative, else predict H 0 ( x) Baum s algorithm seems appropriate for our model because it does not explicitly try to query examples near the boundary. In fact, ....

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

No context found.

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In David S. Touretzky, editor, Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, San Mateo, CA, 1989.

No context found.

A. Blum and R. Rivest. Training a 3-node neural net is NP-complete. In Advances in Neural Information Processing Systems I, pages 494--501, 1989.

No context found.

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, 1989.

No context found.

A. Blum and R. L. Rivest, Training a 3-node neural net is NP-complete, Advances in Neural Information Processing Systems, pp. 494-501 Morgan Kaufmann, (1989).

No context found.

A. Blum and R. L. Rivest. Training a 3-node neural net is NP-Complete. In David S. Touretzky, editor, Advances in Neural Information Processing Systems I, pages 494--501. Morgan Kaufmann, San Mateo, CA, 1989.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC