systems Abstract. The technology for building knowledge-based systems by inductive inference from examples has been demonstrated successfully in several practical applications. This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail. Results from recent studies show ways in which the methodology can be modified to deal with information that is noisy and/or incomplete. A reported shortcoming of the basic algorithm is discussed and two means of overcoming it are compared. The paper concludes with illustrations of current research directions. 1.

In this paper we establish a mathematical framework in which we develop measures for determining the contribution of individual features to the performance of a classifier. Corresponding to these measures, we design metrics that allow estimation of the importance of features for a specific multi-layer perceptron neural network. It is shown that all measures constitute lower bounds for the correctness that can be obtained when the feature under study is excluded and the classifier rebuilt. We also present a method for pruning input nodes from the network such that most of the knowledge encoded in its weights is retained. The proposed metrics and the pruning method are validated with a number of experiments with artificial classification tasks. The experiments indicate that the metric called replaceability results in the tightest error bounds. Both this metric and the metric called expected influence result in good rankings of the features. (c) 1998 Elsevier Science Ltd. All rights reserved.

10 The Author 13 Acknowledgements 15 1 Introduction 16 1.1 Definition of Learning : : : : : : : : : : : : : : : : : : : : : : : : 16 1.2 The objectives of ML : : : : : : : : : : : : : : : : : : : : : : : : : 17 1.3 Approaches taken so far : : : : : : : : : : : : : : : : : : : : : : : 18 1.4 Motivation for the project : : : : : : : : : : : : : : : : : : : : : : 20 1.5 The Structure of the Thesis : : : : : : : : : : : : : : : : : : : : : 21 2 Theory of Inductive Learning 22 2.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 22 2.2 Induction as a Search : : : : : : : : : : : : : : : : : : : : : : : : : 23 2.2.1 The Goal: Hypothesis : : : : : : : : : : : : : : : : : : : : 24 2.2.2 The Search Space: Hypothesis Space : : : : : : : : : : : : 24 2.2.3 The operators : : : : : : : : : : : : : : : : : : : : : : : : : 26 2.3 Approaches : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 27 2.3.1 Statistical Classification : : : : : : : : : : : : : : : : : : : 27...

Automated inductive learning is a vital part of machine intelligence and the design of intelligent agents. A useful formalization of inductive learning is the model of PAC-learnability. Nevertheless, the ability to learn every target concept expressible in a given representation, as required in the PAC-learnability model, is highly demanding and leads to many negative results for interesting concept classes. A new model of learnability, called Universal Learnability or U-learnability, recently has been proposed as a less demanding, average-case variant of PAC-learnability. This paper uses the U-learnability model to analyze a top-down decision tree induction algorithm. Specifically, this paper proves that an idealized variant of the well-known decision tree learning algorithm CART---one of the most successful existing machine learning algorithms---is a U-learner under a natural set of assumptions regarding target hypotheses. (The motivation and description of these assumptions is best...

This rescarch was supportcd by Contract N00014-83-K-0074, NR 154-505, from thc Personncl and ‘Training