The ultimate goal of data mining is to extract knowledge from massive data. Knowledge is ideally represented as human-comprehensible patterns from which end-users can gain intuitions and insights. Axis-parallel hyper-rectangles provide interpretable generalizations for multi-dimensional data points with numerical attributes. In this dissertation, we study the fundamental problem of rectangle-based discriminative data generalization in the context of several useful data mining applications: cluster description, rule learning, and Nearest Rectangle classification. Clustering is one of the most important data mining tasks. However, most clustering methods output sets of points as clusters and do not generalize them into interpretable patterns. We perform a systematic study of cluster description, where we propose novel description formats leading to enhanced expressive power and introduce novel description problems specifying different trade-offs between interpretability and accuracy. We also present efficient heuristic algorithms for the introduced problems in the proposed formats. If-then rules are

Abstract not found

... better than decision trees do for pattern classification problems, they are often regarded as black boxes, i.e., their predictions cannot be explained as those of decision trees. In many applications, it is desirable to extract knowledge from trained ANNs for the users to gain a better understanding of how the networks solve the problems. A new rule extraction algorithm, called rule extraction from artificial neural networks (REANN) is proposed and implemented to extract symbolic rules from ANNs. A standard three-layer feedforward ANN is the basis of the algorithm. A four-phase training algorithm is proposed for backpropagation learning. Explicitness of the extracted rules is supported by comparing them to the symbolic rules generated by other methods. Extracted rules are comparable with other methods in terms of number of rules, average number of conditions for a rule, and predictive accuracy. Extensive experimental studies on several benchmarks classification problems, such as breast cancer, iris, diabetes, and season classification problems, demonstrate the effectiveness of the proposed approach with good generalization ability.

Abstract: A hybrid system using weightless neural networks (WNNs) and finite state automata is described in this paper. With the use of such a system, rules can be inserted and extracted into/from WNNs. The rule insertion and extraction problems are described with a detailed discussion of the advantages and disadvantages of the rule insertion and extraction algorithms proposed. The process of rule insertion and rule extraction in WNNs is often more natural than in other neural network models. Keywords: weightless neural networks; WNNs; RAM-based neural networks; rule insertion; rule extraction; grammatical inference; automata theory; Boolean expression; bio-inspired computation; hybrid intelligence systems. Reference to this paper should be made as follows: Ludermir, T.B., de Souto, M.C.P. and de Oliveira, W.R. (2009) ‘On a hybrid weightless neural system’, Int. J. Bio-Inspired Computation, Vol. 1, Nos. 1/2, pp.93–104.

Evolving model trees for mining data sets with continuous-valued classes

The present paper introduces a new destructive algorithm for rule extraction based on a trained neural network. The degree of complexity of neural network increases exponentially as a factor of the numbers of input and hidden nodes. Therefore, the dimensionality of the trained neural network is reduced by using a proposed destructive algorithm to extract only the most effective values of the input attributes which have higher impact on the output result for each class. Thus, the searching efficiency is highly increased and the computation is dramatically reduced for extracting rules. The generated rules from the proposed model are fired through two levels for each class. As for the first level, it deals with each individual effective input value, and the second level is concerned with each possible conjunction of the effective input values. Moreover, the proposed model extracts the strongest rules which represent a large number of instances from the database by adjusting the similarity measure threshold value. Finally, the proposed model is evaluated on different public-domain datasets and compared with standard learning models from WEKA, then the results assert that the set of rules extraction from the proposed method is more accurate and concise compared with those obtained by the other models.