Abstract. Storing and using specific instances improves the performance of several supervised learning algorithms. These include algorithms that learn decision trees, classification rules, and distributed networks. However, no investigation has analyzed algorithms that use only specific instances to solve incremental learning tasks. In this paper, we describe a framework and methodology, called instance-based learning, that generates classification predictions using only specific instances. Instance-based learning algorithms do not maintain a set of abstractions derived from specific instances. This approach extends the nearest neighbor algorithm, which has large storage requirements. We describe how storage requirements can be significantly reduced with, at most, minor sacrifices in learning rate and classification accuracy. While the storage-reducing algorithm performs well on several realworld databases, its performance degrades rapidly with the level of attribute noise in training instances. Therefore, we extended it with a significance test to distinguish noisy instances. This extended algorithm's performance degrades gracefully with increasing noise levels and compares favorably with a noise-tolerant decision tree algorithm.

Selective induction techniques perform poorly when the features are inappropriate for the target concept. One solution is to have the learning system construct new features automatically; unfortunately feature construction is a difficult and poorly understood problem. In this paper we present a definition of feature construction in concept learning, and offer a framework for its study based on four aspects: detection, selection, generalization, and evaluation. This framework is used in the analysis of existing learning systems and as the basis for the design of a new system, citre. citre performs feature construction using decision trees and simple domain knowledge as constructive biases. Initial results on a set of spatial-dependent problems suggest the importance of domain knowledge and feature generalization, i.e., constructive induction. 1 Introduction Good representations are often crucial for solving difficult problems in AI. Finding suitable problem representations, however, ...

A framework for the construction of new features for hard classification tasks is discussed. The approach brings together ideas from the fields of machine learning, computational geometry, and pattern recognition. Two heuristics for evaluation of newly-constructed features are proposed, and their statistical significance verified. Finally, it is shown how the proposed framework can be used to combine techniques for selection of representative examples with techniques for construction of new features, in order to solve difficult problems in learning from examples. 1. Introduction. The problem of new terms, also known as the constructive induction problem, has long been considered a source of difficulty in machine learning (Dietterich, 1982). Simple classifiers using only the primitive features of description have limited learning capabilities. For example: (i) Single-layered neural networks can realize only those class dichotomies, where the classes are linearly separable in the featur...

A well-known fundamental limitation of selective induction algorithms is that when tasksupplied attributes are not adequate for, or directly relevant to, describing hypotheses, their performance in terms of prediction accuracy and/or theory complexity is poor. One solution to this problem is constructive induction. It constructs, by using task-supplied attributes, new attributes that are expected to be more appropriate than the task-supplied attributes for describing the target concepts. This thesis focuses on constructive induction with decision trees as the theory description language. It explores: (1) novel approaches to constructing new binary attributes using existing constructive operators, and (2) novel methods of constructing new nominal and new continuous-valued attributes based on a newly proposed constructive operator. The thesis investigates a fixed rule-based approach to constructing new binary attributes for decision tree learning. It generates conjunctions from producti...

Aesthetics plays a major role in real design. To date, aesthetics is mostly ignored in studies dealing with computational design support systems. Probably, the main reason for this omission is that aesthetics is tightly associated with art which is perceived to be beyond the capabilities of computational techniques. The paper outlines a model for the incorporation of aesthetic judgment in design. It shows how important aesthetic criteria that follow the rationalistic and the romanticist movements of aesthetics, can be represented, refined, and used in design. Bridger, a system that assists in the preliminary design of cable-stayed bridges, implements a preliminary version of this model. Several examples of designs generated by Bridger are discussed for demonstrating the scope and potential of the model. The most significant investigations in bridge aesthetics deal primarily with fundamental philosophical and psychological problems or direct aesthetic analysis of existing bridges, prov...

Abstract not found

The use of expert systems is becoming more and more widespread, making the need for appropriate machine learning techniques more acute to help ease the knowledge aquisition bottleneck. Additionally, the increasing number of large databases offers a vast potential for the automatic generation of new knowledge by machines and its communication to people in a comprehensible form. In response to these events, this paper provides an overview of current machine learning work with a particular emphasis on rule induction techniques. Firstly we provide a summary of existing rule induction techniques, including descriptions of the ID3 and AQ algorithms. Secondly, we review recent developments in rule induction technology which overcome some of the practical limitations of these basic algorithms including noise handling, probabilistic classification, large data sets and incremental learning. Finally, we describe the state of current research in machine learning and the directions in which it is heading, addressing the difficult problems of constructive induction and representation change.

In real-world domains a concept to be learned may be unwieldy and the environment may be less than ideal. One combination of difficulties occurs if the concept is probabilistic and the learning situation is dynamic. In this case, the data may be noisy and biased. These difficulties arise when learning evaluation functions, which can be considered as concepts. A representative problem, the fifteen puzzle, is used to test six different learning systems: some that fit, count, or partition data in instance, space; some that optimize measures derived from data in hypothesis space; and some that perform combinations of such procedures. These six systems are described, tested, and analyzed. From quantitative differences in several experiments, we extract specific properties. By combining two or three kinds of techniques, we gauge the extent to which they complement each other. Combinations of strengths can overcome difficulties in domains that are simultaneously probabilistic, dynamic, noisy, and biased. 1.

To date, a comprehensive, Yahoo-like hierarchy of topics has yet to be offered for the domain of news. The Yahoo approach of managing such a hierarchy --- hiring editorial staff to read documents and correctly assign them to topics --- is simply not practical in the domain of news. Far too many stories are written and made available online everyday. While many Machine Learning methods exist for organising documents into topics, these methods typically require a large number of labelled training examples before performing accurately. When managing a large and ever-changing topic hierarchy, it is unlikely that there would be enough time to provide many examples per topic. For this reason, it would be useful to identify extra information within the domain of news that could be harnessed to minimise the number of labelled examples required to achieve reasonable accuracy. To this end, the notion of a semi-labelled document is introduced. These documents, which are partially labelled by th...

This is a brief annotated bibliography that I wanted to make available to the attendees of my Machine Learning tutorial at the 1995 AI & Statistics Workshop. These slides