Baum, E. B., Haussler, D. 1989. What size net gives valid generalization?, Neural Computation, 1:151-160.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... given in the task, and we will say that the model built by the network gets a good generalization property if it is close to the target function OE, according to a given metric on the set of functions fI Og.Lower bounds on p = jT j in order to ensure a good generalization have been derived in [BH89] Since these bounds grow with the size of the network, a better generalization of a given task T should be achieved by a smaller network. Therefore, the aim of all the constructive training methods is to build small networks realizing the task. Feedforward neural networks of predetermined ....

Eric B. Baum and David Haussler. What size net gives valid generalization ? Neural Computation, 1(1):151--160, 1989.

....types of artificial neural network. A similar observation was made by Marchand et al. 19, 20] who construct a cascade network from a threshold decision list. So another way of bounding the growth function of threshold decision lists is to use this fact in combination with some known bounds [8, 4] for the growth functions of linear threshold networks. This gives a similar, though slightly looser, upper bound. To bound the growth function of the subclass consisting of s level threshold functions, we use a result from [1] which shows that the number of ways in which a set S of m points can ....

E. Baum and D. Haussler. What size net gives valid generalization? Neural Computation, 1(1):151--160, 1989.

....if h k (x) If there is no index satisfying this property, then the output is simply equal to 1Q (the example is not assigned to any category) As a consequence, M fat H ( is exactly the graph dimension of the MLP. This dimension can be bounded from above using the approach described in [3]. To that end, the notion of growth function must preliminary be introduced. De nition 11 (Growth function) Let F be a set of binary valued functions f on X . For s m 2 X , let F (s m ) be the number of di erent functions corresponding to restrictions of functions in F to s m . Then the ....

....hand, the proof of the following lemma is straightforward. Lemma 1 MLP (m) w (m) Q(Q 1) 2 where w is the growth function of any of the hidden units, neglecting once more the state 0. This lemma can be used to bound the graph dimension of the MLP in a way similar to what was done in [3]. It su ces then to bound w (m) in terms of the corresponding fat shattering dimension, which can be done thanks to a generalized Sauer s lemma. This constitutes the last step of the decomposition, since sharp bounds on the fat shattering dimension of linear classi ers are available. The ....

E.B. Baum and D. Haussler. What size net gives valid generalization? Neural Computation, 1:151160, 1989.

....to memorize individual points rather than learn the general patterns. In the case of neural networks, the number of weights, which is inexorably linked to the number of hidden layers and neurons, and the size of the training set (number of observations) determine the likelihood of overfitting [2,18]. The greater the number of weights relative to the size of the training set, the greater the ability of the network to memorize idiosyncrasies of individual observations. As a result, generalization for the validation set is lost and the model is of little use in actual forecasting. Therefore, ....

E.B. Baum and D. Haussler, What size net gives valid generalization?, Neural Computat. 6 (1989) 151-160. 235

....in which estimates of a posteriori probabilities, or of the best guess classification (where the best guess might not be very accurate) are required. VC PAC analysis has been applied to feedforward networks of binary threshold elements, to which we direct interested readers attention [2]. The study of classifier generalization is typically viewed as a problem of determining the optimal parameterization for a classifier, given some fixed number of training samples. Interest in the functionalform of the objective function used to train the classifier has rarely gone beyond ....

E. B.Baum and D. Haussler, "What Size Net Gives Valid Generalization?" Neural Computation, vol. 1, pp. 151-160, spring, 1989.

....network is too large, after a long training phase it will learn the given set correctly, but will badly generalize due to learning data overfitting. Therefore, the goal when training a network is to find a topology large enough to learn the mapping and as small as possible to generalize correctly [1,2]. The previous procedure, based on the training of several topologies without taking advantage of previous network learning, is heavy and time consuming. This work was supported in part by the Consiglio Nazionale delle Ricerche of Italy and in part by the Ministero della Pubblica Ist uzione of ....

E.B. BAUM, D. HAUSSLER, "What size net gives valid generalization?", Neural Computation, No. 1, 1989, pp. 151-160.

....the given set This work was supported in part by the C.N.R. of Italy under the project Sistemi Elettronici Avanzati Reft Neurali and in part by the Ministero dell Univ. e della Ricerca Scientifica e Tecnologica of Italy. correctly, but will badly generalize due to learning data overfitting [2]. Therefore, the goal when training a network is to find a topology large enough to learn the mapping and as small as possible to generalize correctly. Two possible approach to produce networks with correct topologies have been proposed: 1) start with a small network and grow additional synapses ....

E.B: Baum, D. Haussler. "What size net gives valid generalization?",Neural Compuat.,No.l.1989. pp. 151-160.

....acceptable solution. In fact, if the network is too small, the input output mapping cannot be learned with satisfactory accuracy. Conversely, if the network is too large, after a long training phase it will learn the given set correctly, but will badly generalize due to learning data overfitting [2]. Therefore, the goal when training a network is to find a topology large enough to learn the mapping and as small as possible to generalize correctly. Two possible approach to produce networks with correct topologies have been proposed: 1) start with a small network and grow additional synapses ....

E.B. Baum, D. Haussler, "What size net gives valid generalization?", Neural Computation, No.1, 1989, pp.151-160.

....an acceptable solution. In fact, if the network is too small, the input output mapping cannot be learned with satisfactory accuracy. Conversely, if the network is too large, after a long training phase it will learn the given set correcdy, but will badly generalize due to learning data overfitting [2,3]. Therefore, the goal when training a network is to find a topology large enough to learn the mapping and as small as possible to generalize correctly. Two possible approach to produce networks with correct topologies have been proposed: 1) start with a small network and grow additional synapses ....

E.B. Baum, D. Haussler, "What size net gives valid generalization?", Neural Computation, No. 1, 1989, pp. 151-160.

....an acceptable solution. In fact, ff the network is too small, the inputoutput mapping cannot be learned with satisfactory accuracy. Conversely, if the network is too large, after a long txaining phase it will learn the given set correctly, but will badly generalize due to learning data overfitting [2,3]. Therefore, the goal when training a network is to find a topology large enough to learn the mapping and as small as possible to generalize correctly. This agrees with the notion of Minimum Description Length [4] which states that a simple network whose description needs a small number of bits ....

E.B. BAUM, D. HAUSSLER, "What size net gives valid generalization?", Neural Computation, No. 1, 1989, pp. 151-160.

....is in terms of the perceptron weights in the network) The complexity of each perceptron in this result is related to the proportion of training examples which are close to the perceptron threshold. The measure of complexity suggested by existing VC bounds for threshold networks (see, for example, [10]) is related to the number of weights in the network. If a network classi es most examples with a large margin and the network s perceptrons have few examples close to threshold, then our measure of complexity can be considerably smaller. For details see [50] A binary mask perceptron, as ....

E. B. Baum and D. Haussler. What size net gives valid generalization? Neural Computation 1, pages 150-160, 1989.

....in terms of the weights assigned to the perceptrons) The complexity of each perceptron in this result is related to the proportion of training examples which are close to the perceptron threshold. The measure of complexity suggested by existing VC bounds for threshold networks (see, for example, [3]) is related to the number of weights in the network. If a network classifies most examples with a large margin and the network s perceptrons have few examples close to threshold, then our measure of complexity can be considerably smaller. In It0] Golea et al. bound the generalization error of a ....

....VCdim(E1) ind. Substituting this bound into Theorem 2.1 and simplifying gives the result. This result is best understood in relation to existing results for single hiddenlayer threshold networks. By bounding the VC dimension of the class of single hidden layer threshold networks (see [3]) we can apply a standard VC result (using a comparable rate to that of Theorem 3.1) to show that roughly (ignoring constants and log terms) PD(yf(x) O) 2P , yf(x) O) q . In [2] Bartlett presents alternative bounds on the generalization error of more general classes of neural ....

E. B. Baum and D. Haussler. What size net gives valid generalization? Neural Computation 1, pages 150-160, 1989.

No context found.

Baum, E., D. Haussler, "What size net gives valid generalization?", to appear in I.E.E.E. Conference on Neural Information Processing Systems, Denver, CO, 1988.

No context found.

Baum, E. B., Haussler, D. 1989. What size net gives valid generalization?, Neural Computation, 1:151-160.

No context found.

E. B. Baum and D. Haussler, "What size net gives valid generalization ?", in Advances in Neural Information Processing 1, D. S. Touretzky, Ed. Morgan Kaufmann, 1989, pp. 81--90.

No context found.

Baum, E. B., & Haussler, D. (1989). What size net gives valid generalization? Neural Computation, 1, 151--160.

No context found.

E. B. Baum and D. Haussler, What size net gives valid generalization? Neural Computation 1 (1989), 151-160.

No context found.

Eric Baum and David Haussler. What size net gives valid generalization? Neural Computation, 1(1):151--160, 1989.

No context found.

E.B. Baum, and D. Haussler, What Size Net Gives Valid Generalization? Neural Computation, 1 (1989) 151--160.

No context found.

E. B. Baum and D. Haussler, What size net gives valid generalization? Neural Computation 1 (1989), 151-160.

No context found.

E. B. Baum and D. Haussler. What Size Net Gives Valid Generalization ? Neural Computation, 1(1):151-160, 1989.

No context found.

E. B. Baum and D. Haussler, "What size net gives valid generalization ?", in Advances in Neural Information Processing 1, D. S. Touretzky, Ed. Morgan Kaufmann, 1989, pp. 81--90.

No context found.

E. B. Baum and D. Haussler. What size net gives valid generalization? Neural Computation, 1:151--160, 1989.

No context found.

E. Baum and D. Haussler. What Size Net Gives Valid Generalization?. Neural Computation, 1(1), 1989: 151--160.

No context found.

E. B. Baum and D. Haussler. What Size Net Gives Valid Generalization ? Neural Computation, 1(1):151-160, 1989.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC