E. Fiesler. Comparative bibliography of ontogenic neural networks. In Proceedings of the International Conference on Artificial Neural Networks, pages 793-- 796, 1994.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....pruning algorithms. The second approach, which corresponds to constructive algorithms, starts with a small network and then grows additional hidden units and weights until a satisfactory solution is found. Review for pruning algorithms can be found in [t] while that for constructive algorithms in [2], 3] 4] The constructive approach has a number of advantages over the pruning approach. Firstly, for constructive algo rithms, it is straightforward to specify an initial network 1, whereas for pruning algorithms, one does not know in practice how big the initial network should be. Secondly, ....

E. Fiesler, "Comparative bibliography of ontogenic neural net- works," in Proceedings of the International Conference on Artificial Neural Networks, Sorrento, Italy, May 1994, vol. 1, pp. 793-796.

.... of classifiers or function approximators there exist especially designed algorithms which build networks with simple nodes, limited fan in, and a local connectivity structure [3, 2] These algorithms are constructive, i.e. they explicitly build up a network architecture adapted to the problem [4, 8]. In contrast, classical neural net al..gorithms like backpropagation fix the architecture and only train the weights. Many of the VLSI friendly algorithms have been developed with regard to their efficiency, but not with regard to their generalization performance. Especially in the case of noisy ....

Fiesler, E. (1994) Comparative Bibliography of Ontogenic Neural Networks. ICANN94, Proceedings, Springer, London, 793--796.

....En effet, historiquement, les rseaux rtropropagation du gradient tels qu ils ont t proposs dans le milieu des annes 1980 [Rumelhart et al. 86] Le Cun 87] sont essentiellement monoblocs et homognes. Les dfauts de cette approche ont t bien identifis#: difficult pour choisir la taille du rseau [Fiesler 94] Jutten Chentouf 95] structure arbitraire pas ncessairement adapte la tche, mauvaises capacits de gnralisation [Alpaydin 91] Le Cun 89] difficults d apprentissage [Fahlman Lebiere 91] Haykins 94] dfauts qui ont amen les chercheurs proposer d autres approches#: l approche modulaire ....

Emile Fiesler. Comparative Bibliography of Ontogenic Neural Networks. Proc. ICANN, p. 793-796, Sorrento, Italie, 26-29 mai 1994.

....network using EKF learning algorithm (RANEKF) was proposed by [ Kadirkamanathan and Niranjan, 1993 ] The previous version of the IncNet [ Kadirkamanathan, 1994 ] is a RAN EKF network with statistically controlled growth criterion. For more exhaustive description of ontogenic neural networks see [ Fiesler, 1994 ] The EKF equations can be written as follows: e n = yn f(xn ; pn 1 ) dn = #f(xn ;pn 1 ) #pn 1 R y = Rn d T n Pn 1dn kn = Pn 1dn R y pn = pn 1 e n kn Pn = I knd T n ]Pn 1 Q 0 (n)I (15) The su#xes n 1 and n denotes the priors and posteriors. pn consists of all adaptive ....

E. Fiesler. Comparative bibliography of ontogenic neural networks. In Proceedings of the International Conference on Artificial Neural Networks, 1994.

.... Techniques for reducing the number of parameters are greedy constructive learning [7] pruning [5, 12, 14] or weight sharing [18] Techniques for reducing the size of each parameter dimension are regularization, such as weight decay [13] and others [25] or early stopping [17] See also [8, 20] for an overview and [9] for an experimental comparison. Early stopping is widely used because it is simple to understand and implement and has been reported to be superior to regularization methods in many cases, e.g. in [9] 1.2 The basic early stopping technique In most introductory papers ....

Emile Fiesler (efiesler@idiap.ch). Comparative bibliography of ontogenic neural networks. (submitted for publication), 1994.

....an alternative to evolutionary techniques, which have been successfully used for the design and topology adaptation of artificial neural networks, though as the result of an off line adaptation process. Such networks with modifiable topologies have been called ontogenic artificial neural networks [6]. Basically, they make use of two mechanisms that may modify the structure of the network: growth and pruning. Ontogenic artificial neural A. P erez Uribe is supported by the Centre Suisse d electronique et Microtechnique CSEM, Neuchatel, Switzerland. networks include supervised growing ....

E. Fiesler. Comparative bibliography of ontogenic neural networks. Proceedings of the International Conference on Artificial Neural Networks (ICANN'94), 1994.

....pruning algorithms. The second approach, which corresponds to constructive algorithms, starts with a small network and then grows additional hidden units and weights until a satisfactory solution is found. Review for pruning algorithms can be found in [1] while that for constructive algorithms in [2], 3] 4] The constructive approach has a number of advantages over the pruning approach. Firstly, for constructive algorithms, it is straightforward to specify an initial network 1 , whereas for pruning algorithms, one does not know in practice how big the initial network should be. Secondly, ....

E. Fiesler, "Comparative bibliography of ontogenic neural networks, " in Proceedings of the International Conference on Artificial Neural Networks, Sorrento, Italy, May 1994, vol. 1, pp. 793--796.

....k K) corresponds to class C k and learns the posterior probability of class C k given the input x. A number of other constructive methods that can only be applied to classification problems (such as [2, 15, 23, 62] will not be discussed here. Interested readers may consult the short surveys in [21, 70]. This paper will review both purely constructive procedures and procedures that have constructive components for feedforward neural networks. The rest of this paper is organized as follows. In Section 2, general issues on constructive approximation will be discussed. A taxonomy of constructive ....

E. Fiesler. Comparative bibliography of ontogenic neural networks. In Proceedings of the International Conference on Artificial Neural Networks, volume 1, pages 793--796, Sorrento, Italy, May 1994.

.... both strategies to adapt the size of the network [dBZN94, Def95] It is not the purpose of this paper to discuss in details the various facets of all these training algorithms remodeling the size of the network; comparative studies based on a wide selection of these methods can be found in [Fie94, KY95] However, we will recall in section 4 the main features of some of these algorithms in order to locate our methods in their context. A formal definition of the neural model considered in this study will be given in section 2. The heuristic technique used to solve the discrete optimization ....

Emile Fiesler. Comparative bibliography of ontogenic neural networks. In Maria Marinaro and Pietro G. Morasso, editors, Proceedings of the International Conference on Artificial Neural Networks (ICANN 94), volume 1, pages 793--796, London, U.K., 1994. Springer-Verlag.

.... the two strategies to adapt the size of the network [dBZN94, Def95] It is not the purpose of this paper to discuss in details the various facets of all these training algorithms remodeling the size of the network; a comparative study based on a wide selection of these methods can be found in [Fie94] However, we will recall in section 4 the main features of some of these algorithms in order to locate our methods in their context. A formal definition of the neural model considered in this study will be given in section 2. The heuristic technique used to solve the discrete optimization ....

Emile Fiesler. Comparative bibliography of ontogenic neural networks. In Maria Marinaro and Pietro G. Morasso, editors, Proceedings of the International Conference on Artificial Neural Networks (ICANN 94), volume 1, pages 793--796, London, U.K., 1994. Springer-Verlag.

....criterion. Another very good example, derived from MLP network, is the Cascade Correlation algorithm [4] Feature Space Mapping (FSM) system is the system which joins two strategies: growing and pruning, see [2] for more information. For more exhaustive description of ontogenic neural network see [5]. 2 The IncNet Pro Framework Fast EKF: We introduce new fast version of the EKF learning algorithm, described in [1] The EKF was chosen because it can estimate not only adaptive parameters, but also some others values which will be used in novelty criterion and in pruning. Covariance matrix Pn ....

E. Fiesler. Comparative bibliography of ontogenic neural networks. In Proceedings of the International Conference on Artificial Neural Networks, 1994.

....on changing the strength of their interconnections according to a learning algorithm. However, the lack of knowledge in determining the appropriate number of layers, the number of neurons per layer, and how they will be interconnected, limits such ability. The so called ontogenic neural networks [4] try to overcome this problem by offering the possibility of dynamically modifying their topology. Other potential advantages of ontogenic neural networks are improved generalization and better implementation optimization (for size and or execution speed) Neural networks with dynamic topologies ....

E. Fiesler: "Comparative Bibliography of Ontogenic Neural Networks" Proceedings of the International Conference on Artificial Neural Networks (ICANN'94)

....algorithms for competitive networks. The first one is a model derived from Kohonen s self organizing maps [7] in which the network is viewed as a dynamic population of competing neurons. The latter is a self organizing network with automatic size and structure, called Growing Cell Structures. In [3], Fiesler presents a comparative bibliography of a special class of neural networks that automatically adapt its topology to a problem including the above models and calling them ontogenic neural networks. Most of the above algorithms are computationally intensive and it is difficult to adapt them ....

E. Fiesler. Comparative bibliography of ontogenic neural networks. Proceedings of the International Conference on Artificial Neural Networks (ICANN'94), 1994.

....Another very good example, derived from MLP network, is the Cascade Correlation algorithm[5, 6] Feature Space Mapping(FSM) system is the system which joins two strategies: growing and pruning, see [3] for more information. For more exhaustive description of ontogenic neural network see [7]. 2 The IncNet Pro Framework EKF: We decided to use the EKF algorithm [2] because it can estimate not only the parameters (weights, biases, etc. but also some others values such as the output of the network for a given input vector, uncertainty in the expected output (Ry ) matrix P which ....

E. Fiesler. Comparative bibliography of ontogenic neural networks. In Proceedings of the International Conference on Artificial Neural Networks, 1994.

....vast field of evolutionary computation and genetic connectionism. A large biography on evolutionary design of neural architectures may be found in the Internet [105] The discussion here is focused mainly on the MLP optimization methods. Many such methods have been elaborated upon in literature [74, 76, 78, 85, 93, 106, 107, 108]. The constructive neural models that modify network topologies during the learning NEURAL COMPUTING SURVEYS 2, XXX YYY, 1998, http: www.icsi.berkeley.edu jagota NCS 18 process [1, 2, 5, 3] are sometimes called ontogenic networks [108] An ontogenic network has some advantages in comparison to ....

.... elaborated upon in literature [74, 76, 78, 85, 93, 106, 107, 108] The constructive neural models that modify network topologies during the learning NEURAL COMPUTING SURVEYS 2, XXX YYY, 1998, http: www.icsi.berkeley.edu jagota NCS 18 process [1, 2, 5, 3] are sometimes called ontogenic networks [108]. An ontogenic network has some advantages in comparison to the classical MLP: its architecture is not designed ad hoc or by a trial and error experiments, its performance is usually better, the computing time is reduced because adding neurons one after another requires little re training, and ....

[Article contains additional citation context not shown here]

E. Fiesler, Comparative bibliography of ontogenic neural networks, In Proceedings of the International Conference on Artificial Neural Networks, Springer Verlag, pp. 793-796, 1994.

....on the problems tested. 1 Introduction Multi layer feedforward networks have been popularly used in many pattern classification and regression problems. While standard back propagation performs gradient descent in the weight space of a network with fixed topology, constructive procedures [4] start with a small network and then grow additional hidden units and weights until a satisfactory solution is found. A particularly successful subclass of constructive procedures [3, 6] proceeds in a layer by layer manner. First, the weights feeding into the new hidden unit are trained (input ....

E. Fiesler. Comparative bibliography of ontogenic neural networks. In Proceedings of the International Conference on Artificial Neural Networks, volume 1, pages 793--796, Sorrento, Italy, May 1994.

....that results is close to having an optimal structure. There has been little research into training networks which are sparse throughout the course of their training. A classification of networks whose structures do adapt during the course of their training is to be found in a paper by Fiesler [1]. Few of these networks, however, appear to be designed in order to allow supervised learning to be applied to very high dimensional problems. One interesting case of a sparse neural network being applied to a highdimensional problem is the area of phoneme probability estimation [8] In this paper ....

E. Fiesler. Comparative bibliography of ontogenic neural networks. In Proceedings of the International Conference on Artificial Neural Networks (ICANN 94), pages 793--796, 1994.

No context found.

E. Fiesler (1994), Comparative Bibliography of Ontogenic Neural Networks, in these proceedings (ICANN'94).

No context found.

E. Fiesler. Comparative bibliography of ontogenic neural networks. In Proceedings of the International Conference on Artificial Neural Networks, pages 793-- 796, 1994.

No context found.

Emile Fiesler #e#esler@idiap.ch#. Comparative bibliography of ontogenic neural networks. #submitted for publication#, 1994.

No context found.

E. Fiesler, "Comparative bibliography of ontogenic neural networks," in Proceedings of the International Conference on Artificial Neural Networks, pp. 793--796. Sorrento, Italy, 1994.

No context found.

E. Fiesler, "Comparative bibliography of ontogenic neural networks, " in Proceedings of the International Conference on Artificial Neural Networks, Sorrento, Italy, May 1994, vol. 1, pp. 793--796.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC