Abstract

This paper continues work reported at ML'94 on the use of the Minimum Description Length Principle with non-probabilistic theories. A new encoding scheme is developed that has similar benefits to the adhoc penalty function used previously. The scheme has been implemented in c4.5rules and empirical trials on 25 real-world datasets reveal a small but useful improvement in classification accuracy. 1 INTRODUCTION When classifiers are induced from data, the resulting theories are commonly interpreted as functions from attribute values to classes rather than to class distributions. So, for example, we talk of the accuracy of the learned classifier on unseen cases, measured as the percentage of such cases for which the classifier predicts the actual class. Such theories and their interpretation will be described as categorical, although the synonym deterministic is also in common use. A concern when learning in real-world domains is that the theory should not overfit the data because overl...

Citations

3143	Classification and Regression Trees - Breiman, Friedman, et al. - 1983
800	Fast effective rule induction - Cohen - 1995
66	Complexity regularization with applications to artificial neural networks - BARRON - 1991