Although backpropagation neural networks generally predict better than decision trees do for pattern classification problems, they are often regarded as black boxes, i.e., their predictions cannot be explained as those of decision trees. In many applications, more often than not, explicit knowledge is needed by human experts. This work derives symbolic representations from a neural network to make epxlicit each prediction of the network. An algorithm is proposed and implemented to extract symbolic rules from neural networks.

An algorithm for extracting rules from a standard three-layer feedforward neural network is proposed. The trained network is first pruned not only to remove redundant connections in the network, but more importantly, to detect the relevant inputs. The algorithm generates rules from the pruned network by considering only a small number of activation values at the hidden units. If the number of inputs connected to a hidden unit is sufficiently small, then rules that describe how each of its activation values is obtained can be readily generated. Otherwise, the hidden unit will be split and treated as output units, with each output unit corresponding to an activation value. A hidden layer is inserted and a new subnetwork is formed, trained, and pruned. This process is repeated until every hidden unit in the network has a relatively small number of input units connected to it. Examples on how the proposed algorithm works are shown using real-world data arising from molecular bio...

Although backpropagation neural networks generally predict better than decision trees do for pattern classification problems, they are often regarded as black boxes, i.e., their predictions are not as interpretable as those of decision trees. This paper argues that this is because there has been no proper technique that enables us to do so. With an algorithm that can extract rules 1, by drawing parallels with those of decision trees, we show that the predictions of a network can be explained via rules extracted from it, thereby, the network can be understood. Experiments demonstrate that rules extracted from neural networks are comparable with those of decision trees in terms of predictive accuracy, number of rules and average number of conditions for a rule; they preserve high predictive accuracy of original networks.

Neural networks have been shown to be a promising tool for forecasting financial time series. Several design factors significantly impact the accuracy of neural network forecasts. These factors include selection of input variables, architecture of the network, and quantity of training data. The questions of input variable selection and system architecture design have been widely researched, but the corresponding question of how much information to use in producing high-quality neural network models has not been adequately addressed. In this paper, the effects of different sizes of training sample sets on forecasting currency exchange rates are examined. It is shown that those neural networks---given an appropriate amount of historical knowledge ---can forecast future currency exchange rates with 60 percent accuracy, while those neural networks trained on a larger training set have a worse forecasting performance. In addition to higher-quality forecasts, the reduced training set sizes reduce development cost and time.

Neural networks have received a great deal of attention over the last few years. They are being used in the areas of prediction and classification; areas where regression models and other related statistical techniques have traditionally been used. In this paper, we discuss neural networks and compare them to regression models. We start by exploring the history of neural networks. This includes a review of relevant literature on the topic of neural networks. Neural network nomenclature is then introduced and the backpropagation algorithm, the most widely used learning algorithm, is derived and explained in detail. A comparison between regression analysis and neural networks in terms of notation and implementation is conducted to aid the reader in understanding neural networks. We compare the performance of regression analysis with that of neural networks on two simulated examples and one example on a large data set. We show that neural networks act as a type of nonparametric regression...

Abstract. Ontology matching, the task of determining relations that hold among terms of two different ontologies, is a key issue in the Semantic Web and other related fields. In order to compare the behaviour of different ontology matching systems, the Ontology Alignment Evaluation Initiative (OAEI) has established a periodical controlled evaluation that comes in a yearly event. We present here our participation in the 2008 initiative. Our schema-based alignment algorithm compares each pair of ontology terms by, firstly, extracting their ontological contexts up to a certain depth (enriched by using transitive entailment) and, secondly, combining different elementary ontology matching techniques (e.g., lexical distances and vector space modelling). Benchmark results show a very good behaviour in terms of precision, while preserving an acceptable recall. Based on our experience, we have also included some remarks about the nature of benchmark test cases that, in our opinion, could help improving the OAEI tests in the future. 1 Presentation of the system In [7] we presented a system that analyzes a keyword-based user query, in order to automatically extract and make explicit, without ambiguities, its semantics. Firstly, it discovers and extracts candidate senses (expressed as ontology terms)

Artificial neural networks have been used to support applications across a variety of business and scientific disciplines during the past years. Artificial neural network applications are frequently viewed as black boxes which mystically determine complex patterns in data. Contrary to this popular view, neural network designers typically perform extensive knowledge engineering and incorporate a significant amount of domain knowledge into artificial neural networks. This paper details heuristics that utilize domain knowledge to produce an artificial neural network with optimal output performance. The effect of using the heuristics on neural network performance is illustrated by examining several applied artificial neural network systems. Identification of an optimal performance artificial neural network requires that a full factorial design with respect to the quantity of input nodes, hidden nodes, hidden layers, and learning algorithm be performed. The heuristic methods discussed in this paper produce optimal or near-optimal performance artificial neural networks using only a fraction of the time needed for a full factorial design. Keywords: Artificial neural networks; Heuristics; Input vector; Hidden layer size; ANN learning method; Design. Heuristics Principles for the Design of Artificial Neural Networks - Page 3 1.

Current market research data analysis techniques inadequately deal with the huge amounts of information being captured for examination, as a result, significant relationships in the data remain undetected. This paper describes the application of a hybrid expert system /neural network to the task of finding significant events in a market research data base. The neural network classifies trends in the time series data, and the expert system uses these classifications, other knowledge of market research analysis and external events which influence the time series, to infer the significance of the data. The developed system is compared to a system consisting of a linear regression program and an expert system. The neural system produced results with 100% precision and 86% recall on 104 test cases and outperformed the regression system which achieved 95% precision and 72% recall. Both systems were able to perform analysis of the test data in under 5 minutes. The manual analysis of the same ...

One of the challenges of financial research is to develop models that are capable of explaining and forecasting market price movements and returns.Agent based models focus directly on the underlying structure of the market. The basic idea is, that the market price dynamics arises from the interaction of many individual agents. Approaching financial markets in this manner, one starts off with the modeling of the agents´ decision making schemes on the microeconomic level of the market. Thereafter, market price changes can be determined on the macroeconomic level by a superposition of the agents´ buying and selling decisions. The aim of a (micro-)economic model is to explain market prices by a detailed causal analysis of the agents´ decision making behavior. The market price results from an aggregation of the agents´ decisions. Remarkably, agent-based financial markets provide a new explanatory framework supplementing the traditional economic concepts of equilibrium theory and efficient markets. Such a supplementing framework is needed, because in real-world financial markets the underlying assumptions of equilibrium or efficient market theory are often violated.As we will show, neural networks allow the integration of the decision behavior of individual economic agents into a market model. Based on the perspective of interacting agents, the resulting market model allows us to capture the underlying dynamics of financial markets, to fit real-world financial data, and to forecast future market price movements.In addition, we point out that neural networks allow to set up a joint framework of econometric model building. Besides the learning from data, one may integrate prior knowledge about the underlying dynamical system and first principles into the modeling. These elements are incorporated into the neural networks in form of architectural enhancements. This way of model building helps to overcome the drawbacks of purely data driven approaches.

Interest in the application of neural networks as tools for decision support has been growing in recent years. A major drawback often associated with neural networks is the difficulty in understanding the knowledge represented by a trained network. This paper describes an approach that can extract symbolic rules from neural networks. We illustrate how the approach successfully extracted rules from a data set collected from a survey of the service sectors in the United Kingdom. The extracted rules were then used to distinguish between organizations using computers from those that do not. The classification scheme based on these rules was used to identify specific segments of a market for promoting adoption of information technology. The extracted rules are not only concise but also outperform discriminant analysis in terms of predictive accuracy. Keywords. Backpropagation algorithm; neural networks; symbolic rules; IT adoption Original submittal: Sept 6, 1996. First response: July 24...