We review the problems associated with neural networks learning from noisy and/or ambiguous training data and propose a simple procedure that appears to alleviate these problems. By making use of the readily available output error information, a network is able to choose the correct output targets from sets of possibilities and generate new targets if any of the correct ones appear to be missing from the given training data.

We present a new class of neural network models of reading aloud based on Sejnowski & Rosenberg’s NETtalk. Unlike previous models, they are not restricted to mono-syllabic words, require no complicated inputoutput representations such as Wickelfeatures and require no preprocessing to align the letters and phonemes in the training data. The best cases are able to achieve perfect performance on the Seidenberg & McClelland training corpus (which includes many irregular words) and in excess of 95 % on a standard set of pronounceable non-words. Evidence is presented that relate the output activation error scores in the model to naming latencies in humans. Several possible accounts of developmental surface dyslexia are identified and on various forms of damage the models exhibit symptoms similar to acquired surface dyslexia. However, their inability to account for lexical decision, the pseudohomophone effect and phonological dyslexia indicate that we will still need to introduce an additional lexical/semantic route before we have a complete model of reading aloud. Nevertheless, the models ’ simplicity, performance and room for improvement make them a promising basis for the graphemephoneme conversion route of a realistic dual route model of reading.

Interest in the application of neural networks as tools for decision support has been growing in recent years. A major drawback often associated with neural networks is the difficulty in understanding the knowledge represented by a trained network. This paper describes an approach that can extract symbolic rules from neural networks. We illustrate how the approach successfully extracted rules from a data set collected from a survey of the service sectors in the United Kingdom. The extracted rules were then used to distinguish between organizations using computers from those that do not. The classification scheme based on these rules was used to identify specific segments of a market for promoting adoption of information technology. The extracted rules are not only concise but also outperform discriminant analysis in terms of predictive accuracy. Keywords. Backpropagation algorithm; neural networks; symbolic rules; IT adoption Original submittal: Sept 6, 1996. First response: July 24...

A three-layered neural network that optimally self-adjusts the number of hidden layer units is proposed. The network combines two techniques: 1) Unit fusion which enables an efficient pruning of the redundant units. 2) Linear transformations applied to the chosen hidden layer unit pair output and a modified back-propagation training rule for gradual fusion to reduce pruning shocks. The network was applied to a character recognition problem and it adjusted itseft to a minimal configuration at high rate.

The default multilayer neural network topology is a fully interlayer connected one. This simplistic choice facilitates the design but it limits the performance of the resulting neural networks. The best-known methods for obtaining partially connected neural networks are the so called pruning methods which are used for optimizing both the size and the generalization capabilities of neural networks. Two of the most promising pruning techniques have therefore been selected for a comparative study. It is shown that these novel techniques are hampered by having numerous user-tunable parameters, which can easily nullify the benefits of these advanced methods. Finally, based on the results, conclusions about the execution of experiments and suggestions for conducting future research on neural network pruning are drawn. Keyworks: neural network, pruning, parameters, neural network optimization, network size, generalization INTRODUCTION Various neural network pruning techniques have been de...

This paper introduces a pruning algorithm with tridiagonal symmetry constraints for feedforward neural network design. The algorithm uses a reflection transform applied to the input--hidden weight matrix in order to reduce it to its tridiagonal form. The designed FANN structures obtained by applying the proposed algorithm are compact and symmetrical. Therefore, they are well suited for efficient hardware and software implementations. Moreover, the number of the FANN parameters is reduced without a significant loss in performance. We illustrate the complexity and performance of the proposed algorithm by applying it as a solution to a nonlinear regression problem. We also compare the results of our proposed algorithm with those of the Optimal Brain Damage algorithm. EDICS: SP 6.1.5 This work was supported by the Natural Sciences and Engineering Research Council of Canada under contract #06P--0187668. 1 Introduction Feedforward neural network (FANN) design has lately attracted...

I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. ii The work reported in the thesis consists of two parts. One part is concerned with the development of a feature selection method based on statistical significance test, which can be generally used in any supervised pattern classification. The other part applies this proposed feature selection method to conduct proteomic pattern analysis for prostate cancer detection. For a given classification problem, we need to determine a set of relevant features to generate a classifier. In real-world problems, many features in initial feature set are usually irrelevant to the classification task and redundant with each other, which will increase the computational complexity and reduce the recognition rate. The task of feature selection is to choose a small feature subset in order to achieve better classification performance. As such,

Automatic nonlinear-system identification is very useful for various disciplines including, e.g., automatic control, mechanical diagnostics, and financial market prediction. This paper describes a fully automatic structural and weight learning method for recurrent neural networks (RNN). The basic idea is training with residuals, i.e., a single hidden neuron RNN is trained to track the residuals of an existing network before it is augmented to the existing network to form a larger and, hopefully, better network. The network continues to grow until either a desired level of accuracy or a preset maximal number of neurons is reached. The method requires neither guessing of initial weight values nor the number of neurons in the hidden layer from users. This new structural and weight learning algorithm is used to find RNN models for a two-degree-of-freedom planar robot, a Van der Pol oscillator and a Mackey-Glass equation using their simulated responses to excitations. The algorithm is able to find good RNN models in all three cases. 1.

...................................................................................................................... xiv CHAPTERS 1 INTRODUCTION .......................................................................................................... 1 1.1 Background .............................................................................................................. 1 1.2 Present Research...................................................................................................... 3 1.2.1 Modeling of Highway Bridges ........................................................................ 3 1.2.2 Modeling of Live Loads.................................................................................. 4 1.2.3 Neural Network Predictions of Live Load Positions ...................................... 6 1.3 Literature Review .................................................................................................... 7 1.3.1 Modeling of Highway Bridges ...............

It is well known that finding the dimension of the neural network which best solves a given problem, may be difficult and cumbersome. A possible approach is to use an oversized network and then prune off some connections that, when set to zero, leads to the smallest increase of an error measure at the output. Several methods has been proposed in the last years in particular for the MLP. In this paper a new method is presented to dynamically adapt the topology of a neural network by pruning off connections in the early stage of the learning process. Some experimental results are also presented on the 2- region classification problem..