Tin-Yau Kwok and Dit-Yan Yeung. Constructive feedforward neural networks for regression problems: A survey. HKUST-CS95 43, Department of Computer Science, Hong Kong University of Science and Technology, Clear Water Bay,Kowloon, Hong Kong, 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... strategies to adapt the size of the network [dBZN94, Def95] It is not the purpose of this paper to discuss in details the various facets of all these training algorithms remodeling the size of the network# comparative studies based on a wide selection of these methods can be found in [Fie94, KY95] However, we will recall in section 4 the main features of some of these algorithms in order to locate our methods in their context. A formal definition of the neural model considered in this study will be given in section 2. The heuristic technique used to solve the discrete optimization ....

Tin-Yau Kwok and Dit-Yan Yeung. Constructive feedforward neural networks for regression problems: A survey. HKUST-CS95 43, Department of Computer Science, Hong Kong University of Science and Technology, Clear Water Bay,Kowloon, Hong Kong, 1995.

.... of several function approximation techniques throughout B Learn II [59, 33, 17, 65, 34, 3, 21, 48, 49, 52, 50] lead to the selection of neural networks based on local receptive fields [54] such as Radial Basis Function Networks (RBFs) 62] Such networks can be built from training data [54, 57, 3, 43], which is extremely important in a setting that asks for automatization of the learning phase. Additionally, these networks do also allow for directly assessing the knowledge that is available with respect to a particular situation. i.e. by checking the activation of the individual clusters ....

T.-Y. Kwok and D.-Y. Yeung. Constructive feedforward neural networks for regression problems: A survey. Technical Report HKUST-CS95-43, Hong Kong University of Science and Technology, Department of Computer Science, 1995.

....generalization performance and training time of NNs. Current research mostly concentrates on the optimal setting of initial weights [2, 3] optimal learning rates and momentum [4, 5, 6, 7] finding optimal NN architectures using pruning techniques [8, 9, 10, 11, 12, 13] and construction techniques [14, 15, 16], sophisticated optimization techniques [17, 18, 19, 20, 21, 22] and adaptive activation functions [23, 24, 25] This paper presents an alternative approach to improve generalization and training time, i.e. active learning using sensitivity analysis. Standard error back propagating NNs are ....

Kwok, T-Y., Yeung, D-Y.: Constructive Feedforward Neural Networks for Regression Problems: A Survey, Technical Report HKUST-CS95-43, Department of Computer Science, The Hong Kong University of Science & Technology, 1995.

....we shall use to solve this application is divided in three main tasks: database compilation and analysis, neural model selection and its implementation. In our analysis we have considered a wide range of neural models, encompassing classical (i.e. models with fixed structure) as well as evolutive [Kwok et al. 1995] (i.e. those able to find the proper network structure for a given task) algorithms. 2 Definition of the problem Vending machines based on automatic coin recognizers constitute a high volume market niche, due to the large amount of potential applications (like product dispensers, automatic ....

Y. Kwok, D.- Y. Yeung, Constructive Feedforward Neural Networks for Regression Problems: A Survey, Technical Report HKUST-CS95-43, Hong Kong University of Science and Technology , 1995.

....models. More recently, other developments in the field have been brought about with more strict comparisons between the different methods proposed [12] and with a more theoretical study of the properties associated with constructive networks, from the viewpoint of general regression problems [11]. Some other models have also been proposed associated with other feedforward networks than the multilayer perceptron such as RBF networks [1, 9, 17] In accordance with the objectives followed in this work, in the next section it is described the general ideas behind the cascade correlation model ....

T. Kwok and D. Yeung. Constructive feedforward neural networks for regression problems: A survey. Technical ReportHKUST-CS95-43,DepartmentofComputerScience, HongKongUniversityofScienceandTechnology,1995.

....instance, as given in Sanger (1991) Sanger, Sutton, and Matheus (1992) and Shin and Ghosh (1995) Due to the global character of these learning methods, the danger of negative interference is quite large. Additional references on constructive learning for regression can be found in the survey by Kwok and Yeung (1995). The idea of the mixture of experts in Jacobs et al. 1991) and hierarchical mixtures of experts in Jordan and Jacobs (1994) is related to RFWR as the mixture of experts approach looks for similar partitions of the input space, particularly in the version of Xu et al. 1995) Ormeneit and Tresp ....

Kwok, T.-Y., & Yeung, D.-Y. (1995). "Constructive feedforward neural networks for regression problems: A survey." Technical Report HKUST-CS95-43, Department of Computer Science, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong.

....scheme of the upstart algorithm. More recently, other developments in the field have been brought about with more strict comparisons between the different methods proposed [Littmann and Ritter, 1996] and with a more theoretical study of the properties associated with constructive networks [Kwok and Yeung, 1995]. 3 The Cascade correlation Architecture The cascade correlation network [Fahlman and Lebiere, 1990] was created with the objective of minimizing two of the most important problems found in the MLP: the slow pace of the learning process and the need for a priori definition of the number of ....

Kwok, T., Yeung, D. Constructive Feedforward Neural Networks for Regression Problems: A Survey. TR HKUST-CS95-43, Dep. of Comp. Science, Hong Kong Univ. of Sc. and Technology.

....implementations, small networks will reduce the required chip size. Several methods for which the choice of initial network topology is less critical have been proposed in the literature. Examples are early stopping methods [Finnoff 93] explicit regularization [Bishop 95] network growing [Kwok 95], and network pruning [Reed 93] The latter will be considered in this chapter. The basic idea of pruning is to start training a network that is considered sufficiently big to ensure convergence. When convergence is reached certain weights neurons are removed, and the network is retrained. This ....

T. -Y. Kwok and D. -Y. Yeung. Constructive Feedforward Neural Networks for Regression Problems: A Survey. Technical report HKUST-CS95-43, Department of Computer Science, Hong Kong University of Science and Technology, 1995.

....(Mostafa, 1992; Simard et al. 1992) The importance of the VC dimension is the source of a huge amount of research that is devoted to the evolution of appropriately sized networks. This research can be classified as pruning algorithms (LeCun, 1990; Mozer, 1989; Prechelt, 1995) growing approaches (Kwok, 1995), or hybrid methods (Salomon, 1994; Schiffmann, 1990) All approaches consider a training set that has to be known in advance. This problem does not occur in biological brains. From the very beginning, the brain s size is very large (and rather shrinking) and even with an initially small amount of ....

Kwok, T.-Y., Yeung, D.-Y. (1995). Constructive Feedforward Neural Networks for Regression Problems: A Survey. Tech. report HKUST-CS95-43, Hong Kong University of Science and Technology.

....approach adds a few hidden units and continues training. The training growing cycle is repeated until a certain stopping criterion is met. An example for this approach is given by the cascade16 correlation learning architecture (Fahlman Lebiere, 1990) For a thorough discussion, for instance, Kwok and Yeung (1995). The pruning approach starts with training a sufficiently large network. After the network has learned the given task, certain connections and or hidden units are removed. Most methods assign a relevance or salience to each connection unit and pruning is done with respect to these parameters. ....

....network is too large, training time and pruning are unnecessarily long. Starting with a too small network, the pruning approach fails. 2) Adding hidden units from time to time might results in a network that is able to solve the task. But stopping the growing process too late (see, for example, Kwok Yeung, 1995), result in too large networks, which might yield poor generalization. 3) Separating training from topology optimization requires a stopping criterion for the training process. An inappropriate choice slows down the overall convergence speed. 4) The hybrid approach proposed by Schiffmann and ....

Kwok, T.-Y., & Yeung, D.-Y. (1995). Constructive Feedforward Neural Networks for Regression Problems: A Survey. Technical Report HKUST-CS95-43, anonymous ftp, ftp://ftp.cs.ust.hk/pub/techreport/95/tr95-43.ps.gz.

....instance, as given in Sanger (1991) Sanger, Sutton, and Matheus (1992) and Shin and Ghosh (1995) Due to the global character of these learning methods, the danger of negative interference is quite large. Additional references on constructive learning for regression can be found in the survey by Kwok and Yeung (1995). The idea of the mixture of experts in Jacobs et al. 1991) and hierarchical mixtures of experts in Jordan and Jacobs (1994) is related to RFWR as the mixture of experts approach looks for similar partitions of the input space, particularly in the version of Xu et al. 1995) Ormeneit and Tresp ....

Kwok, T.-Y., & Yeung, D.-Y. (1995). "Constructive feedforward neural networks for regression problems: A survey." Technical Report HKUST-CS95-43, Department of Computer Science, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong.

No context found.

T.-Y. Kwok and D.-Y. Yeung, "Constructive feedforward neural networks for regression problems: A survey," Technical Report HKUST-CS95-43, 1995.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC