by Javed A. Aslam, Scott E. Decatur
ftp://das-ftp.harvard.edu/techreports/tr-17-94.ps.gz
Add To MetaCart
Abstract:
The statistical query learning model can be viewed as a tool for creating (or demonstrating the existence of) noise-tolerant learning algorithms in the PAC model. The complexity of a statistical query algorithm, in conjunction with the complexity of simulating SQ algorithms in the PAC model with noise, determine the complexity of the noise-tolerant PAC algorithms produced. Although roughly optimal upper bounds have been shown for the complexity of statistical query learning, the corresponding noisetolerant PAC algorithms are not optimal due to inefficient simulations. In this paper we provide both improved simulations and a new variant of the statistical query model in order to overcome these inefficiencies. We improve the time complexity of the classification noise simulation of statistical query algorithms. Our new simulation has a roughly optimal dependence on the noise rate. We also derive a simpler proof that statistical queries can be simulated in the presence of classification noise. This proof makes fewer assumptions on the queries themselves and therefore allows one to simulate more general types of queries.
Citations
|
1364
|
A theory of the learnable
– Valiant
- 1984
|
|
536
|
Learnability and the Vapnik-Chervonenkis Dimension
– Blumer, Ehrenfeucht, et al.
- 1989
|
|
453
|
The strength of weak learnability
– Schapire
- 1990
|
|
298
|
Boosting a weak learning algorithm by majority
– Freund
- 1995
|
|
223
|
On the method of bounded differences
– McDiarmid
- 1989
|
|
222
|
Fast probabilistic algorithms for Hamiltonian circuits and matchings
– Angluin, Valiant
- 1979
|
|
201
|
Efficient noise-tolerant learning from statistical queries
– Kearns
- 1993
|
|
183
|
Learning from noisy examples
– Angluin, Laird
- 1988
|
|
175
|
A general lower bound on the number of examples needed for learning
– Ehrenfeucht, Haussler, et al.
- 1989
|
|
126
|
Learning in the presence of malicious errors
– KEARNS, LI
- 1993
|
|
108
|
Learning disjunctions of conjunctions
– VALIANT
- 1985
|
|
80
|
Weakly learning DNF and characterizing statistical query learning using Fourier analysis
– Blum, Furst, et al.
- 1994
|
|
73
|
Computational learning theory: Survey and selected bibliography
– Angluin
- 1992
|
|
46
|
An improved boosting algorithm and its implications on learning complexity
– Freund
- 1992
|
|
42
|
General bounds on statistical query learning and PAC learning with noise via hypothesis boosting
– Aslam, Decatur
- 1993
|
|
35
|
Statistical queries and faulty PAC oracles
– Decatur
- 1993
|
|
31
|
General bounds on the number of examples needed for learning probabilistic concepts
– Simon
- 1993
|
|
19
|
Learning from Good and Bad Data. Kluwer international series in engineering and computer science
– Laird
- 1988
|
|
10
|
On the sample complexity of weak learning
– Goldman, Kearns, et al.
- 1990
|
|
9
|
Algorithmic Learning of Formal Languages and Decision Trees
– Sakakibara
- 1991
|
