E. B. Baum, Y. Lyuu, "The Transition to Perfect Generalization in Perceptrons", Neural computation 3, pp. 386-401.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....trainable free parameters in the network. In particular, these authors were interested in training using binary valued weights (quantised Sigma1) between the inputs and hidden layer and weights with value 1 between the hidden layer and the output. Apart from having good generalisation properties [9] such weight values are straightforward to implement in hardware [26] Though binary valued weights would appear to be very restrictive it is important to remember that they will give 2 N evenly distributed hyper16 4 4 2 2 1 1 1 1 2 2 3 3 Output Hidden layer Weights between hidden layer ....

E.B. Baum and Y-D. Lyuu. The transition to perfect generalization in perceptrons. Neural Computation, 3:386--401, 1991.

....= 0 (i.e. origin centered halfspaces) under the uniform distribution on the unit sphere in R n : 6. 1 PREVIOUS WORK The problem of learning an unknown origin centered halfspace in R n given access to examples drawn uniformly from the unit sphere has been the subject of considerable research [7, 8, 16, 22, 28, 34, 39]. Long [28] proved that any algorithm which learns an origin centered halfspace to accuracy ffl under the uniform distribution must use at least Omega Gamma n ffl ) examples. Long also showed [29] that by applying Vaidya s linear programming algorithm [40] it is possible to learn to accuracy ....

E. B. Baum and Y-D. Lyuu. The transition to perfect generalization in perceptrons. Neural Computation, 3:386-401, 1991.

....values of # 1 , # 2 , # 3 as in figure 3. reached zero error with probability approaching 1 in the thermodynamic limit for scaled sample size greater than 1.448. This bound on the critical value was known from the work of Gardner and Derrida (1989) and extended to the case of boolean inputs by Baum, Lyuu and Rivin (1991; 1992) Here we are actually giving a bound on the entire learning curve, and the behavior of our bound is very similar in shape to learning curves obtained in both simulations and non rigorous replica calculations from statistical physics (Engel Fink, 1993; Gyorgyi, 1990; Seung et al. 1992; ....

Baum, E.B., & Lyuu, Y.-D. (1991). The transition to perfect generalization in perceptrons. Neural Comput., 3:386--401.

.... D we consider is the uniform distribution on the unit sphere (or any other radially symmetric distribution) Despite the voluminous literature on learning perceptrons in general (see the work of Minsky and Papert [17] for a partial bibliography) and with respect to this distribution in particular [23, 2, 6], no efficient noise tolerant learning algorithm has been given previously. Here we give a very simple and efficient algorithm for learning from statistical queries (and thus an algorithm tolerating noise) The sketch of the main ideas is as follows: for any vector v 2 R n , the error of v ....

Eric B. Baum and Yuh-Dauh Lyuu. The transition to perfect generalization in perceptrons. Neural Computation, 3:386-- 401, 1991.

....the number of labels that it needs to know. Baum[Bau91] proposed a learning algorithm that uses membership queries to avoid the intractability of learning neural networks with hidden units. His algorithm is proved to work for networks with at most 4 hidden units, and there is experimental evidence[BL91] that it works for larger networks. However, when Baum and Lang tried to use this algorithm to train a network for classifying handwritten characters, they encountered an unexpected problem[BL92] The problem was that many of the images generated by the algorithm as queries did not contain any ....

E. B. Baum and Y.-D. Lyuu. The transition to perfect generalization in perceptrons. Neural Comput., 3:386--401, 1991.

....algorithm must have reached zero error with probability approaching 1 in the thermodynamic limit for scaled sample size greater than 1:448. This bound on the critical value was known from the work of Gardner and Derrida [10] and extended to the case of boolean inputs by Baum, Lyuu and Rivin [2, 15]. Here we are actually giving a bound on the entire learning curve, and the behavior of our bound is very similar in shape to learning curves obtained in 5 The designation Ising refers to the Sigma1 constraint, which is present in the original Ising model of magnetism with N interacting ....

E. B. Baum and Y.-D. Lyuu. The transition to perfect generalization in perceptrons. Neural Comput., 3:386-- 401, 1991.

....domain distribution P is known a priori. Given these stronger assumptions, many researchers have shown that both rational and exponential learning curves are possible. For example, exponential convergence has been demonstrated in many distribution specific analyses of particular concept spaces [GM93, BL91, PS90, SSSD90, SST91], and rational convergence has been demonstrated for other spaces [OH91] 5 This paper shows how, in a general way, this dichotomy can still be revealed under much weaker assumptions. We also draw a clean boundary between these two modes of convergence in terms of a simple structural property of ....

E. B. Baum and Y.-D. Lyuu. The transition to perfect generalization in perceptrons. Neural Computation, 3:386--401, 1991.

....algorithm must have reached zero error with probability approaching 1 in the thermodynamic limit for scaled sample size greater than 1:448. This bound on the critical value was known from the work of Gardner and Derrida [12] and extended to the case of boolean inputs by Baum, Lyuu and Rivin [2, 19]. Here we are actually giving a bound on the entire learning curve, and the behavior of our bound is very similar in shape to learning curves obtained in both simulations and non rigorous replica calculations from statistical physics [14, 28, 25, 9] 6 In Figure 11, we graph the difference of ....

E. B. Baum and Y.-D. Lyuu. The transition to perfect generalization in perceptrons. Neural Comput., 3:386-- 401, 1991.

No context found.

E. B. Baum, Y. Lyuu, "The Transition to Perfect Generalization in Perceptrons", Neural computation 3, pp. 386-401.

....Algorithm has been shown more effective than all known competitors 1 . The robustness of our genetic algorithm allows us to extend our results to in various directions. The problem of learning a half space with integer weights from examples drawn from a uniform distribution has been studied in [28, 29, 9, 5]. We show that this problem can be directly mapped into a noisy ASP problem, yielding some solution techniques. Alternatively consider the problem a Classifier System [12] faces in learning from interaction with a complex environment. In complex environments one may have little a priori ....

....for the GA. However the MFA algorithm, because it is oblivious, can be defeated by problems with less symmetry that the GA can solve. 9 Learning Half Spaces with integer weights In this section we consider the Ising Perceptron Problem , aka the problem of learning half spaces of integer weight [28, 29, 9, 5]. Let t 2 f1; Gamma1g N be an N dimensional vector and let H t be the halfspace (x; t) 0. We are given a number of samples x 1 ; x s in the N dimensional unit ball classified according to whether they are in H t or not in H t . The samples were chosen uniformly and independently in the ....

E. B. Baum, Y. Lyuu, "The Transition to Perfect Generalization in Perceptrons", Neural computation 3, pp. 386-401.

No context found.

Eric B. Baum and Yuh-Dauh Lyuu. The transition to perfect generalization in perceptrons. Neural Computation, 3:386--401, 1991.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC