Applied Research/Communications Lab.
|
Abstract:
A memory-based learning system is an extended memorymanagement system that decomposes the input space either statically or dynamically into subregions for the purpose of storing and retrieving functional information. The main generalization techniques employed by memory-based learning systems are the nearest-neighbor search, space decomposition techniques, and clustering. Research on memory-based learning is still in its early stage. In particular, there are very few rigorous theoretical results regarding memory requirement, sample size, expected performance, and computational complexity. In this paper, we propose a model for memory-based learning and use it to analyze several methods--- ffl-covering, hashing, clustering, tree-structured clustering, and receptive-fields---for learning smooth functions. The sample size and system complexity are derived for each method. Our model is built upon the generalized PAC learning model of Haussler [1] and is closely related to the method of vector quantization in data compression. Our main result is that we can build memory-based learning systems using new clustering algorithms(Lin and Vitter [2]) to PAC-learn in polynomial time using only polynomial storage in typical situations. Keywords: Memory-based learning, PAC learning, clustering, approximation,
Citations
| 7709 | Computers and Intractability: A Guide to the Theory of NP-Completeness – Garey, Johnson - 1979 |
| 457 | Approximation algorithms for combinatorial problems – Johnson - 1974 |
| 457 | A new polynomial-time algorithm for linear programming – Karmarkar - 1984 |
| 259 | A polynomial algorithm in linear programming – Khachiyan - 1979 |
| 256 | Multivariate adaptive regression splines – Friedman - 1991 |
| 231 | A greedy heuristic for the set-covering problem – Chvatal - 1979 |
| 185 | On the ratio of optimal integral and fractional covers – Lovász - 1975 |
| 113 | Learning with localized receptive fields – Moody, Darken - 1989 |
| 79 | On the complexity of some common geometric location problems – Megiddo, Supowit - 1984 |
| 26 | Worst-case and probabilistic analysis of a geometric location problem – Papadimitriou - 1981 |
| 8 | Programming of interdependent activities, ii, mathematical model. Activity Analysis of Production and Allocation – Dantzig - 1951 |
| 6 | Nearly optimal vector quantization via linear programming – Lin, Vitter - 1992 |
