A new algorithm for neural network pruning is presented. Using this algorithm, networks with small number of connections and high accuracy rates for breast cancer diagnosis are obtained. We will then describe how rules can be extracted from a pruned network by considering only a finite number of hidden unit activation values. The accuracy of the extracted rules is as high as the accuracy of the pruned network. For the breast cancer diagnosis problem, the concise rules extracted from the network achieve an accuracy rate of more than 95 % on the training data set and on the test data set. Keywords. Neural network pruning; penalty function; rule extraction; breast cancer diagnosis. 2 1 Introduction Neural networks techniques have recently been applied to many medical diagnostic problems [1, 2, 4, 5, 11, 22]. Although the predictive accuracy of neural networks is often higher than that of other methods or human experts, it is generally difficult to understand how the network arrives a...

Before symbolic rules are extracted from a trained neural network, the network is usually pruned so as to obtain more concise rules. Typical pruning algorithms require retraining the network which incurs additional cost. This paper presents FERNN, a fast method for extracting rules from trained neural networks without network retraining. Given a fully connected trained feedforward network with a single hidden layer, FERNN first identifies the relevant hidden units by computing their information gains. For each relevant hidden unit, its activation values is divided into two subintervals such that the information gain is maximized. FERNN finds the set of relevant network connections from the input units to this hidden unit by checking the magnitudes of their weights. The connections with large weights are identified as relevant. Finally, FERNN generates rules that distinguish the two subintervals of the hidden activation values in terms of the network inputs. Experimental results show th...

The evolutionary neural network (ENN) is a new system for modeling multifactor data. The strengths of ENN's are that they can extract insignificant predictors, choose the size of the hidden layer and fine tune the parameters needed in training the network. We have used an ENN to predict the biological activities of Dihydrofolate Reductase Inhibitors. As a result, we found that evolutionary neural networks give more accurate predictions than statistical methods and feedforward neural networks. Keywords: Evolutionary neural netwoks, Genetic algorithms, QSAR. 3 1. Introduction The field of classical Quantative Structure Activity Relationships (QSAR), as we know it today, began with the seminal work of Hansch and Fujita[1]. QSAR related biological activity for members of a congeneric series with substituted parameters. QSAR represents an important stage in the development of our understanding of the fundamentals of the processes and factors controlling drug action, and has provided man...

Abstract. Feedforward neural networks represent a well-established computational model, which can be used for solving complex tasks requiring large data sets. When dealing with this kind of problems, the main requirements will be the speed of the learning process and the ability to generalize well the extracted knowledge. To satisfy these demands, adequate initial parameters of the model – like number of layers and number of neurons – are essential. For a given problem, especially the architecture of the model impacts its generalization capabilities. Optimal network architecture speeds up recall and may also improve efficiency of further retraining. Some of the techniques also enable or simplify further knowledge extraction. The main goal of this article is to provide a survey of some existing techniques that optimize architecture of BP-networks.