Abstract

learning Boolean functions, linear-threshold algorithms Abstract. Valiant (1984) and others have studied the problem of learning various classes of Boolean functions from examples. Here we discuss incremental learning of these functions. We consider a setting in which the learner responds to each example according to a current hypothesis. Then the learner updates the hypothesis, if necessary, based on the correct classification of the example. One natural measure of the quality of learning in this setting is the number of mistakes the learner makes. For suitable classes of functions, learning algorithms are available that make a bounded number of mistakes, with the bound independent of the number of examples seen by the learner. We present one such algorithm that learns disjunctive Boolean functions, along with variants for learning other classes of Boolean functions. The basic method can be expressed as a linear-threshold algorithm. A primary advantage of this algorithm is that the number of mistakes grows only logarithmically with the number of irrelevant attributes in the examples. At the same time, the algorithm is computationally efficient in both time and space. 1.

Citations

3340	Pattern Classification and Scene Analysis - Duda, Hart - 1973
1515	A Theory of the Learnable - Valiant - 1984
687	Estimation of dependences based on empirical data - Vapnik - 1982
582	Queries and concept learning - Angluin - 1988
565	Learnability and the Vapnik Chervonenkis dimension - Blumer, Ehrenfeucht, et al. - 1989
517	Generalization as search - Mitchell - 1982
377	Parallel distributed processing: Explorations in the microstructure of cognition - Rumelhart, McClelland - 1986
286	Inductive inference: Theory and methods - ANGLUIN, SMITH - 1983
155	On the Learnability of Boolean Formula - Kearns, Li, et al. - 1987
150	Threshold Logic and its Applications - Muroga - 1971
127	Learning Machines - Nilsson
114	Learning disjunctions of conjunctions - Valiant - 1985
44	Predicting {0, 1}-functions on randomly drawn points - HAUSSLER, LITTLESTONE, et al. - 1994
43	On the prediction of general recursive functions - Barzdin, Freivald - 1972
31	Linear function neurons: structure and training - Hampson, Volper - 1986
29	VALIANT: Recent results on Boolean concept learning - KEARNS, LI, et al. - 1987
9	Quantifying the inductive bias in concept learning - Haussler - 1986
2	Space Efficient Learning Algorithms. Unpublished manuscript - Haussler - 1986
1	The logic of learning: A basis for pattern recognition and for improvement of performance - Banerji - 1985
1	The programmer's guide to the Connection Machine (Technical Report - Slade - 1987