T. Ash. Dynamic node creation in backpropagation networks. Connection Science, 1(4):365--375, 1989.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....della Pubblica Ist uzione of Italy. The a priori knowledge about the particular problem characteristics can only partially alleviate the procedure. A method able to dynamically adapt the network topology to the problem concurrently with the weight adaptation is therefore highly desirable [3]. This property has an interesting reference in the neurobiology field. In fact, as shown by Hubel. and Wiesel during the 60 years, the brain keeps a plasticity over time in several structural and functional details also if its fundamental organization does not change. In particular limited time ....

T. ASH , "Dynamic node creation in backpropagation networks", ICS Report 8901, UCSD Feb. 1989.

....and or neurons until the desired behaviour is retained. The first approach, although it could be efficient since most of the job is performed on networks of small size, can be difficult to realize and control in order to reach network topologies which are optimal in some senses (see for example [3]) The second approach is much more studied and consists of finding a subset of network synaptic weights that, when set to zero, lead to the smallest increase of an error measure at the output. Several methods has been proposed in the last years in particular for the MLP [4,5,6,7,8] However they ....

T. Ash , "Dynamic node creation in backpropagation networks", ICS Report 8901, UCSD Feb. 1989.

....synapses and or neurons until the desired behaviour is mantained. The first approach, although it could be efficient since most of the job is performed on networks of small size, can be difficult to realize and control in order to reach network topologies which are optimal in some senses. In [4] a single, non optimal procedure to add neurons to a MLP is reported. The second approach is much more studied and consists of finding a subset of network synaptic weights that, when set to zero, lead to the smallest increase of an error measure at the output. Several methods has been proposed in ....

T. Ash, "Dynamic node creation in backpropagation networks", ICS Report 8901, UCSD Feb. 1989.

....neurons until the desired behaviour is maintained. The f zrst approach, although it could be efficient since most of the job is performed on networks of small size, can be difficult to realize and control in order to reach network topologies which are optimal in some senses (see for example [5]) The pruning approach is much more studied and consists of finding a subset of network synaptic weights that, when set to zero, leads to the smallest increase of an error measure at the output. Several methods has been proposed in the last years in particular for the MLP. In [6] Sietsma and ....

T. ASH, "Dynamic node creation in backpropagation networks", ICS Report 8901, UCSD Feb. 1989.

....are very simple (only one hidden layer, only one neuron of output) the generalization of their structure can be made in a dynamic way very easily. Some research studies were already undertaken in order to build neural networks dynamically. Those include the dynamic creation of the nodes [1], the cascades correlation algorithm [4] the tilling algorithm [8] the algorithm self organizing [13] and the upstart algorithm [5] These algorithms are used to eliminate the need to determine in advance (before the training of a network) the number of neurons of the hidden layer. This ....

T. Ash. Dynamic node creation in backpropagation networks. Connection Science, 1(4):365-375, 1989.

....of the network, it can be constructed during learning and then reduced later until an optimum solution is gained. Constructive methods (also called growing) start with an input and output layer and add hidden nodes or weights (links) during learning until the network can represent the function [4,5,6]. Reduction (also known as pruning) removes superfluous parts of the network, while still representing the object function [7,8] Incremental networks are useful as they provide the user with less parameters to decide upon before learning has been done. Classically, the number of nodes the ....

Ash T. "Dynamic node creation in backpropagation networks." Connection Science 1989, 1(4):365-375

....was suggested by Tenorio and Lee [TL90] Given a set of possible links to a new neuron, they propose a way to select the best ones to actually connect. Another approach to dynamic ANN construction is adding width to the network rather than depth. For example, the Dynamic Node Creation algorithm [Ash89] starts with a small ANN with a single hidden layer and trains it using back propagation. Whenever the performance of the ANN does not improve fast enough, a new neuron is added to the hidden layer and the process is continued. An interesting twist on such a width increasing algorithm is the ....

T. Ash. Dynamic node creation in backpropagation networks. Connection Science, 1:365--375, 1989.

....1.6 2 mltpx 64 6 4 0.2 0.9 and reported an average CPU time of 20000 sec. in a SUN workstation 3 for the 6 parity problem. Furthermore, these results were obtained after careful optimization of the parameters involved and he simply disregarded all cases where no convergence was obtained. Ash [Ash 89] reports superior convergence results with a constructive algorithm, but the larger examples reported are the 6 parity and 6asymmetry, which are trivially simple for our algorithm. Furthermore, his algorithm failed to find solutions with the minimum number of units even for these smaller problems ....

Ash, Timur "Dynamic Node Creation in Back Propagation Networks", Connection Science 1:4, pp. 365-375, 1989.

....accuracy and rule simplicity (as discussed in Section 1) an appropriate number of hidden units must be determined, and two general approaches have been proposed in the literature. The constructive algorithms start with a few hidden units and add more units as needed to improve network accuracy [5, 6, 7]. The destructive algorithms, on the other hand, start with a large number of hidden units and remove those that are found to be redundant [8] The number of useful input units corresponds to the number of relevant input attributes of the data. Typical algorithms usually start by assigning one ....

T. Ash, \Dynamic node creation in backpropagation networks," Connection Science, vol. 1, no. 4, pp. 365-375, 1989.

....of a given learning task. But estimating the number before the learning task is done is difficult. To meet such a requirement, the actual number of hidden units should be flexible and adjustable to the optimum number during learning. There may be many such kinds of learning systems (for example, Ash, 1989; Hagiwara, 1990) A simple one of flexible networks is the growth network that is constructed by adding hidden units one by one so as to reduce the output error of the network (for example, Fahlman Lebiere, 1990) For speed learning, the main network is fixed during learning; only the added ....

Ash, T. (1989). Dynamic node creation in backpropagation networks. Proceedings of the International Joint Conference on Neural Networks, Washington D. C., June, Vol. II, p. 623.

....[39] learning the inverse kinematic transformations of a robot arm controller [126] the classification of cervical cells [93] and continuous speech recognition [73] 3.3 Node Creation and Node Splitting Algorithms 3.3. 1 Dynamic Node Creation The Dynamic Node Creation method of Ash [4] adds fully connected nodes to the hidden layer of a feed forward neural network architecture trained using Back Propagation. Training starts with a single hidden node and proceeds until the functional mapping is learned (the final error is below a tolerance) or the error ceases to descend and a ....

T. Ash. Dynamic node creation in backpropagation networks. Connection Science, 1:365--375, 1989.

No context found.

T. Ash. Dynamic node creation in backpropagation networks. Connection Science, 1(4):365--375, 1989.

No context found.

Ash, T. (1989). Dynamic node creation in backpropagation networks. Connection Science, 1, 365375. Berliner, H. (1979). The B* tree search algorithm: A best-first proof procedure. Artificial Intelligence, 12, 23-40.

No context found.

Ash, T. (1989). Dynamic node creation in backpropagation networks. Connection Science, 1, 365375. Banerji, R. B. (1980). Artificial intelligence: A theoretical approach. Elsevier.

No context found.

T. Ash, "Dynamic node creation in back-propagation networks, " Connection Science, vol. 1, pp. 365--375, 1989.

No context found.

Ash, T. 1989. Dynamic node creation in backpropagation networks. Connection Science 1: 365--75.

No context found.

Ash, Timur "Dynamic Node Creation in Back Propagation Networks", Connection Science 1:4, pp. 365-375, 1989.

No context found.

T. Ash, "Dynamic node creation in backpropagation networks," Connection Sci., vol. 1, pp. 365--375, 1989.

No context found.

Timur Ash. Dynamic node creation in backpropagation networks. Connection Science, 1(4):365--375, 1989.

No context found.

T. Ash, "Dynamic node creation in backpropagation networks," Connection Sci., vol. 1, no. 4, pp. 365--375, 1989.

No context found.

T. Ash, "Dynamic node creation in back-propagation networks, " Connection Science, vol. 1, pp. 365--375, 1989.

No context found.

T. Ash. Dynamic Node Creation in Backpropagation Networks. Connection Science, 1(4):365--375, 1989.

No context found.

T. Ash, "Dynamic node creation in backpropagation networks H, ICS Report 8901, UCSD Feb. 1989.

No context found.

T. Ash, "Dynamic node creation in backpropagation networks," Connection Science, 4:1,365-375 (1989).

No context found.

Ash, T., "Dynamic Node Creation in Backpropagation Networks", Connection Science, Vol.1, No.4, 1989.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC