Tenorio, M. F. and W. T. Lee, "Self-organizing network for optimum supervised learning", IEEE Transactions on Neural Networks, Vol. 1, No. 1, pp. 100--110, 1990.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....can be made in a dynamic way very easily. Some research studies were already undertaken in order to build neural networks dynamically. Those include the dynamic creation of the nodes [1] the cascades correlation algorithm [4] the tilling algorithm [8] the algorithm self organizing [13] and the upstart algorithm [5] These algorithms are used to eliminate the need to determine in advance (before the training of a network) the number of neurons of the hidden layer. This is very useful because a simpler network having less hidden neurons reduces the complexity of calculations. ....

M. Tenorio and W. Lee. Selforganizing network for optimum supervised learning. IEEE Transactions on Neural Networks, 1(1):100-110, 1990.

....only to the output of the neurons in the previous layer. This can be thought of as an MLP network with new layers inserted between the last hidden layer and the output layer. A further extension to the idea of restricting the connections in cascade correlation was suggested by Tenorio and Lee [TL90] Given a set of possible links to a new neuron, they propose a way to select the best ones to actually connect. Another approach to dynamic ANN construction is adding width to the network rather than depth. For example, the Dynamic Node Creation algorithm [Ash89] starts with a small ANN with a ....

M. F. Tenorio and W. T. Lee. Self-organizing network for optimum supervised learning. IEEE Transactions on Neural Networks, 1(1):100--110, Mar 1990.

....for the required length of the chain needed to obtain good discrimination between functions fitting the data and those modelling the added noise are given, and these are confirmed by experiment. 1 Introduction The self structuring of artificial neural networks has been found to have many benefits [1]. While a large network is required to learn any function, for a specific function defined by the training data at hand, a very much smaller network will often be suitable. This smaller network is also much less likely to overfit the data, and so will generalise more effectively when presented ....

M.F. Tenorio and W.T. Lee. Self-Organizing Network for Optimum Supervised Learning. IEEE Trans. Neural Networks, 1(1):100--110, 1990.

.... 7 J , i 3 l i N W m A NB O 71N Nc d Q i a h S )Ls NA 2C KH C FAH9g ;O E GzH K4Y j d 9 # 3 N o N jK N0l D K GMDH(Group Method of Data Handling) k[Ivakhnenko 71] # 3 N jK b R e j 9 F # C rMQ k a KC5:w N6I=j J I NLdBj , XE 5 l F k[Tenorio 90] # 0J2 G O 3 7 7gE r9nI 9 k a KGMDH HGP(9=B E GA K NE 9g r ;n k #I. T i ,Ds0F 9 k 7 9 F JGMDH 9=B E GA K rSTROGANOFF (STructured Representation On Genetic Algorithms for NOn linear Function Fitting) H8F V # 3 l OB =E2s5 J, O NE 7WE jK HGP K4p E E,1 E C5:w ....

....1 , z 2 NA Br 3. Step3 G N= N; r7o N7hDj G k #= Mh NGMDH O 55,2=K H8F P l k R e j 9 F # C rMQ FG N7A 0 Hz 1 , z 2 N8uJd N7hDj r9T C F [Ivakhnenko 71] # 7 7 J , i F1; K 3 N R e j 9 F # C J A K5 0x 9 k7gE JAH9g ;O E GzH d6I=jCM X N H i C W K rGMDH OM 7 F [Tenorio 90] # 3 l i N LdBj r2r7h 9 k a 9=B E JGA JGP) HGMDH HE 9g 9 k 3 H r;n k #0J2 G O STROGANOFF N4pK 86M K D F bL 9 k # GMDH GM ( i l k40A47A y OLZ N7A 0 GI=8= 5 l k #Nc ( P 3 8(a) NGMDH N2aDx r9M ( k # 3 N l9g N40A47A y O N h KM ( i l k # z 1 = G x1,x2 (x 1 , x 2 ) 19) z ....

[Article contains additional citation context not shown here]

Tenorio, M.F. and Lee, W., Self-organizing Network for Optimum Supervised Learning, IEEE Tr. Neural Networks, vol.1, no.1, 1990

....years, much research have been done on algorithms that dynamically construct neural networks for solving this pattern classification problem. These algorithms include the dynamic node creation [1] the cascade correlation algorithm [2] the tiling algorithm [3] the self organizing neural network [4], and the upstart algorithm [5] These construction algorithms were designed to eliminate the need to determine the number of hidden units prior to training required by backpropagation learning. The aim of these algorithms is to use as few hidden units as possible to solve a given problem. A ....

M.F. Tenorio and W. Lee, "Self-organizing network for optimum supervised learning," IEEE Transactions on Neural Networks, vol.1, no. 1, pp. 100-110, 1990.

....based on the MackeyGlass [67] differential equation dx(t) dt = Gammabx(t) a Delta x(t Gamma T ) 1 x(t Gamma T ) 10 (5) is recognized as a benchmark for comparing the learning and generalization ability of different neural architectures. Following previous studies [12] 68] 69] [70], 71] 72] we generated this time series using the parameters a = 0.2, b = 0.1, and T = 17. As in the studies cited above, the task for the neural network is to predict the value of the time series at point x[t I] from the earlier points (x[t] x[t Gamma D] x[t Gamma 2D] x[t Gamma ....

M. F. Tenorio, "Self-organizing network for optimum supervised learning," IEEE Transactions on Neural Networks, vol. 1, no. 1, pp. 100--110, 1990.

....realized function describes the training data in a least mean squares sense, is used to terminate and evaluate the algorithm. Unfortunately, this approach often leads to suboptimal structures because of its heuristic nature. Another similar approach is the self organizing neural network algorithm [16]. This algorithm constructs a network, chooses the activation functions and adjusts the weights to incrementally build a model of the unknown system using representations selected from a predetermined set of polynomials. The algorithm shares the spirit of group method of data handling type ....

....group method of data handling type algorithms, but the use of a modified minimum description length criterion in conjuction with stochastic search based on simulated annealing for selection of node transfer functions leads to models that are simpler and more accurate. Comparisons with MLP given in [16] for the Mackey Glass chaotic series forecasting problem show that the number of epochs required is less by an order of magnitude. This advantage is more than offset by the long times taken per epoch, largely due to simulated annealing search, which also degrades rapidly in time and quality with ....

[Article contains additional citation context not shown here]

M. F. Tenorio and W. Lee,"Self-Organizing Network for Optimum Supervised Learning," IEEE Trans. on Neural Networks , Vol.1, No.1, pp.100-110, Mar. 1990.

.... It is interesting to notice that the global behavior of this optimization method is comparable with the group method of data handling (GMDH) in which additional terms are incrementally added to the existing polynomial approximator to achieve a minimal description length model of a complex system [9, 34]. The performance of the BGP method on noisy data was tested with the majority problem of 9 inputs. Unlike in the previous experiments where all possible examples are used without noise insertion, we used in each run a training set of 256 examples with 5 noise. This means, on average, 12 or 13 ....

M. F. Tenorio and W. -T. Lee, "Self-Organizing Network for Optimum Supervised Learning," IEEE Transactions on Neural Networks, 1 (1990) 100--110.

....of a one hidden layer network. A fixed connectivity scheme can be a limiting factor if the function being approximated cannot be represented efficiently by a model with the connectivity scheme imposed by the algorithm. Although it is possible to search through connectivity schemes as well [46], such algorithms must incorporate a good heuristic for managing the combinatorial explosion of possibilities. 3. In a recent comparative study, models trained using a constructive approach tended to be less robust [13] Although this study did not include constructively trained MLPs, one of its ....

M.F. Tenorio and W.-T. Lee. Self--organizing network for optimum supervised learning. IEEE Transactions on Neural Networks, 1(1):100--110, 1990.

....rate of change and also the number of epochs trained, is then measured. This cycle repeats again until a satisfactory network solution is found. Note that as learning progresses, the pool may have networks with different numbers of hidden units. 2.1.4. 3 Nondeterministic Search For example, in [97], a new architecture is randomly chosen from the set of possible architectures. An energy criterion, modified from the minimal description length (MDL) 79] information criterion, is then computed for the new network and the question of whether to accept or reject this network is determined via ....

....manner. In [73] new connections may be created by any possible combination of the outputs from the preceding hidden (or input) layer. Candidate hidden units are then added to the network whenever their performances satisfy certain performance measure. In the self organizing neural network [97], candidate hidden units are generated by allowing connections to any input or pre existing hidden unit. A new hidden unit is then randomly selected from this candidate pool and simulated annealing is used to determine whether to accept or reject this addition. The resulting architecture is no ....

[Article contains additional citation context not shown here]

M.F. Tenorio and W.T. Lee. Self-organizing network for optimum supervised learning. IEEE Transactions on Neural Networks, 1(1):100--110, March 1990.

....process repeated. KWOK AND YEUNG: CONSTRUCTIVE ALGORITHMS FOR STRUCTURE LEARNING 5 1 V 2 V 1 2 V E E E 3 3 Fig. 1. Typical state traversal with a single valued state transition mapping. Usually, jV 2 j = jV 1 j 1; jV 3 j = jV 2 j 1, and so on. form of nondeterministic search is used in [49]. Certain search techniques require the definition of an evaluation function that evaluates the desirability of each individual candidate. Usually, this is an estimate of the generalization performance (Section II C) A potential problem is then the increase in time requirement if multiple ....

....of a number of training patterns. Moreover, there is no hard threshold to control the minimum spacing between the RBF units. A major drawback of these algorithms is that their convergence properties are unknown. E. Group Method of Data Handling Constructive algorithms in this category [35] [49] are inspired by the group method of data handling (GMDH) developed by Ivakhnenko [129] The major difference from algorithms described in previous sections is that the state transition mapping is multi valued. Each hidden unit takes a fixed number of incoming connections, but the sources of these ....

[Article contains additional citation context not shown here]

M.F. Tenorio and W.T. Lee, "Self-organizing network for optimum supervised learning," IEEE Transactions on Neural Networks, vol. 1, no. 1, pp. 100--110, Mar. 1990.

No context found.

Tenorio, M. F. and W. T. Lee, "Self-organizing network for optimum supervised learning", IEEE Transactions on Neural Networks, Vol. 1, No. 1, pp. 100--110, 1990.

No context found.

Tenorio, M. F. and W. T. Lee, "Self-organizing network for optimum supervised learning", IEEE Transactions on Neural Networks, Vol. 1, No. 1, pp. 100--110, 1990.

No context found.

M. F. Tenorio and T. W. Lee. Self-organizing network for optimum supervised learning. IEEE Transactions on Neural Networks, 1(1):100--110, 1990.

No context found.

M.F. Tenorio and W. Lee, "Selforganizing network for optimum supervised leathing," IEEE Trans. on Neural Networks, 1:1, 100-110 (1990).

No context found.

Tenorio, M.F. and Lee, W. Self-organizing Network for Optimum Supervised Learning, IEEE Tr. Neural Networks, vol.1, no.1, pp.100-110, 1990 18

No context found.

M. F. Tenorio, "Self-organizing network for optimum supervised learning," IEEE Transactions on Neural Networks, vol. 1, no. 1, pp. 100--110, 1990.

No context found.

Tenorio, M. F. (1990). Self-organizing network for optimum supervised learning. IEEE Transactions on Neural Networks, 1(1):100--110.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC