M. Frean. The upstart algorithm: A method for constructing and training feed-forward neural networks. Technical Report Preprint 89/469, Edinburgh Physics Dept, 1990.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....difficulties is to let the training algorithm modify the topology of the network. A variety of training algorithms adapting the size of the network have been proposed. Some of them, called constructive algorithms, essentially increase the size of the network until the job is fully performed [MN89,Fre90, GM90, SN90, RCE82] while others start from a large networkandtrytopruneit during the training phase [SD88, WHR90, Ree93] Finally, other methods combine both strategies to adapt the size of the network [dBZN94, Def95] It is not the purpose of this paper to discuss in details the various ....

.... algorithm [MN89] and its simplest variant called the tower algorithm [Gal86, Nad89] the decision tree algorithms [GM90, SN90] or the parity machine [ME92, MD89] are typical examples of forward constructive algorithms, while the construction of the network is backward in the upstart method [Fre90] 4.1 Forward methods In a forward method, the network is built layer bylayer from the input to the output. In the present description, we will focus on the case where connections may occur only between two consecutivelayers. In this setting, during the construction of layer h 1, only layer h ....

[Article contains additional citation context not shown here]

Marcus Frean. The upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation, 2(2):198--209, 1990.

....over several layers and interconnection between them. Several methods have been proposed to automatically construct ANNs for reduction in network complexity that is to determine the appropriate number of hidden units, layers, etc. Topological optimization algorithms such as Extentron [9] Upstart [35], Pruning [63] 75] and Cascade Correlation [29] etc. got its own limitations. The interest in evolutionary search procedures for designing ANN architecture has been growing in recent years as they can evolve towards the optimal architecture without outside interference, thus eliminating the ....

....3.2.1.2 Evolutionary Search of Architectures Evolutionary architecture adaptation can be achieved by constructive and destructive algorithms. Constructive algorithms, which add complexity to the network starting from a very simple architecture until the entire network is able to learn the task [35] [56] 59] Destructive algorithms start with large architectures and remove nodes and interconnections until the ANN is no longer able to perform its task [63] 75] Then the last removal is undone. Figure 16 demonstrates how typical neural network architecture could be directly encoded and how ....

Frean M (1990), The upstart algorithm: a method for constructing and training feed forward neural networks, Neural computations Volume 2, pp.198-209.

.... useful for approximating unknown functional relationships between input and output data streams [11] Among these, Schaal and Atkeson proposed a Receptive Field Weighted Regression (RFWR) algorithm with incremental learning ability in [1] This algorithm is related to constructive learning [10] and local function approximation based on the well known radial basis function networks. But with some particular nonparametric regression techniques involved, RFWR is more efficient for incremental function approximation in the sense that it is not necessary to store the training data and ....

Frean, M., "The upstart algorithm: A method for constructing and training feedforward neural networks", Neural Computation, 1990, 2, pp. 198-209.

....way very easily. Some research studies were already undertaken in order to build neural networks dynamically. Those include the dynamic creation of the nodes [1] the cascades correlation algorithm [4] the tilling algorithm [8] the algorithm self organizing [13] and the upstart algorithm [5]. These algorithms are used to eliminate the need to determine in advance (before the training of a network) the number of neurons of the hidden layer. This is very useful because a simpler network having less hidden neurons reduces the complexity of calculations. However, the fact of fixing this ....

M. Frean. The upstart algorithm: a method for constructing and training feedforward neural networks. Neural Computation, 2(2):198-209, 1990.

....over several layers and interconnection between them. Several methods have been proposed to automatically construct ANNs for reduction in network complexity that is to determine the appropriate number of hidden units, layers, etc. Topological optimization algorithms such as Extentron [7] Upstart [3], Pruning [18] and Cascade Correlation [8] etc. got its own limitations. The interest in evolutionary search procedures for designing ANN architecture has been growing in recent years as they can evolve towards the optimal architecture without outside interference, thus eliminating the tedious ....

....8. Global search of optimal architecture (Algorithm 2) Evolutionary architecture adaptation can be achieved by constructive and destructive algorithms. Constructive algorithms starting from a very simple architecture add complexity to the network until the entire network is able to learn the task [3 4, 12]. Destructive algorithms start with large architectures and remove nodes and interconnections until the ANN is no longer able to perform its task [18] Then the last removal is undone. Figure 3 demonstrates how typical neural network architecture could be directly encoded and how the genotype is ....

) Frean, M. (1990): The Upstart Algorithm: A Method For Constructing and Training Feed Forward Neural Networks, Neural computations, pp.198-209.

....algorithm for global search of optimal architecture Evolutionary architecture adaptation can be achieved by constructive and destructive algorithms. Constructive algorithms starting from a very simple architecture add complexity to the network until the entire network is able to learn the task [3 4, 9]. Destructive algorithms start with large architectures and remove nodes and interconnections until the ANN is no longer able to perform its task [15] Then the last removal is undone. Figure 5 demonstrates how a typical neural network architecture could be directly encoded and how the genotype is ....

Frean M.: The Upstart Algorithm: A Method for Constructing and Training Feed Forward Neural Networks, Neural computations Volume 2, pp.198-209, (1990).

....classification. 1) Constructive Learning Using Iterative Weight Update: A number of algorithms that incrementally construct networks of threshold neurons for learning the binary to binary mapping have been proposed in the literature (for example, the tower, pyramid [19] tiling [31] upstart [15], oil spot [29] and sequential [28] algorithms) These algorithms differ in terms of their choices regarding: restrictions on input representation (e.g. binary or bipolar valued inputs) when to add a neuron; where to add a neuron; connectivity of the added neuron; weight initialization for the ....

M. Frean, "The upstart algorithm: A method for constructing and training feedforward neural networks," Neural Comput., vol. 4, pp. 198--209, 1990.

....methods in order to find a good genetic network. In the next section, we will propose a practical algorithm, inspired by the HAKKE system [5, 8] We also note that our problem is very closely related to a classical problem of learning by a perceptron, where some learning algorithms are proposed [4, 6, 7]. 145 Algorithm AK algorithm Given S: set of linear inequalities over variables X begin while S #= # do if there are inequalities in S that contains a single variable then U : x # X x occurs as a single variable in at least one inequalities of S ; pick at random y # U ; F (y) ....

Frean, M. R., The upstart algorithm: a method for constructing and training feedforward neural networks, Neural Comput, 2:198--209, 1990.

....algorithms. With the other approach, the training begins with a minimal network and ends with a satisfactory network size. The algorithms using this approach are referred to as growth or constructive methods. Examples include the cascade correlation leaning architecture[7] upstart algorithm [8], and the tiling algorithm [9] In multi layer feedforward neural networks, a minimal network does not have any hidden layer. 3 Bottom Up Freezing Algorithm Pruning methods start with a large network that is over parameterized and eliminate relatively unimportant nodes or connections. The ....

Frean, M. The upstart algorithm: a method for constructing and training feedforward neural networks. Neural Computation 2(2):198-209, 1990.

....methods in order to find a good genetic network. In the next section, we will propose a practical algorithm, inspired by the HAKKE system [5, 8] We also note that our problem is very closely related to a classical problem of learning by a perceptron, where some learning algorithms are proposed [4, 6, 7]. 145 Algorithm AK algorithm Given S: set of linear inequalities over variables X begin while S 6= OE do if there are inequalities in S that contains a single variable then U : fx 2 X j x occurs as a single variable in at least one inequalities of Sg ; pick at random y 2 U ; F (y) fe 2 ....

Frean, M. R., The upstart algorithm: a method for constructing and training feedforward neural networks, Neural Comput, 2:198--209, 1990.

.... a small network, add units and or weights if necessary during training [8] ffl Destructive learning: Start with a large network delete units and or delete decay weights that have little contribution to learning, e.g. 10] 29] 21] Some of the studies are done by , Marchand et al. 25] Frean [9]. In a study by [42] the network is trimmed by removing unimportant weights and even units. One approach to get rid of unimportant weights is to add a new term representing network complexity to the cost function and let it 13 decay the unnecessary weights to zero. In [16] Ishikawa and Uchida ....

M. Frean, "The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks", Neural Computation, 2, 198, (1990).

....a bias node and one hidden node would be represented as in Figure 5. 9 Input nodes Bias Output node Hidden node Figure 5: A cascade architecture with three inputs, a single hidden node and one output. 2.4 Tree architectures: the Upstart algorithm 2.4. 1 The algorithm The Upstart algorithm [43, 42] is an efficient constructive procedure for building classifier networks. As noted above there are two types of error that can occur at the output node ( 1,0) node quantisation is required as we will see) ffl The output is 1 when it should have been 0. We call these patterns wrongly on. ffl The ....

....to a network with a single hidden layer. Output node Input layer ve ve A B Correctors Figure 9: An example of a possible architecture generated by the Upstart algorithm with two corrector nodes. In constrast to this example the tree will grow asymmetrically for most typical datasets. Frean [43] performed numerical experiments with artificial tasks (N parity, 2 or more clumps , etc) and the generalisation ability of the resulting networks was good and much better than earlier algorithms such as the Tower and Tiling algorithms. However, the algorithm does not appear to have been used ....

[Article contains additional citation context not shown here]

M. Frean. The upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation, 2:198--209, 1990.

No context found.

M. Frean. The upstart algorithm: A method for constructing and training feed-forward neural networks. Technical Report Preprint 89/469, Edinburgh Physics Dept, 1990.

No context found.

Frean, M.: (1990) The upstart algorithm: a method for constructing and training feedforward neural networks, Neural Computation 2: 198-209

No context found.

Frean, M. (1990). The Upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation, 2, 198-209.

No context found.

Frean, M. R., "The upstart algorithm: a method for constructing and training feedforward neural networks", IEEE Transactions on Neural Networks, Vol. 2, pp. 198--209, 1990.

No context found.

Frean, M. (1990). The Upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation, 2, 198-209.

No context found.

Frean, M. R., "The upstart algorithm: a method for constructing and training feedforward neural networks", IEEE Transactions on Neural Networks, Vol. 2, pp. 198--209, 1990.

No context found.

Frean, M. The upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation 2 (1990), 198--209.

No context found.

M. Frean. The upstart algorithm: a method for constructing and training feedforward neural networks. Neural Computation, 2:198--209, 1990.

No context found.

Frean, M. The upstart algorithm: A method for constructing and training feedforward neural networks. Neural Computation 2 (1990), 198--209.

No context found.

Marcus Frean, "The upstart algorithm: A method for constructing and training feedforward neural networks," Neural Computation, vol. 2, pp. 198--209, 1990.

No context found.

Frean, M. (1989). The Upstart Algorithm: A Method for Constructing and Training Feed-Forward Neural Networks. Edinburgh Physics Preprint 89/479, Dept. of Physics, University of Edinburgh.

No context found.

Frean, M. (1989). The Upstart Algorithm: A Method for Constructing and Training Feed-Forward Neural Networks. Edinburgh Physics Preprint 89/479, Dept. of Physics, University of Edinburgh. 3

No context found.

M. Frean, "The upstart algorithm: A method for constructing and training feedforward neural networks," Neural Computation, vol. 2, no. 2, pp. 198--209, 1990.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC