Download:
|
by Rudiger Reischuk, Thomas Zeugmann
in Proc. 16th Int'l Sympos. on Theoretical Aspects of Computer Science, STACS'99
http://www.i.kyushu-u.ac.jp/~thomas/stacs99rzfin.ps.gz
Add To MetaCart
Abstract:
Abstract. We advocate to analyze the average complexity of learning problems. An appropriate framework for this purpose is introduced. Based on it we consider the problem of learning monomials and the special case of learning monotone monomials in the limit and for on-line predictions in two variants: from positive data only, and from positive and negative examples. The well-known Wholist algorithm is completely analyzed, in particular its average-case behavior with respect to the class of binomial distributions. We consider different complexity measures: the number of mind changes, the number of prediction errors, and the total learning time. Tight bounds are obtained implying that worst case bounds are too pessimistic. On the average learning can be achieved exponentially faster. Furthermore, we study a new learning model, stochastic finite learning, in which, in contrast to PAC learning, some information about the underlying distribution is given and the goal is to find a correct (not only approximatively correct) hypothesis. We develop techniques to obtain good bounds for stochastic finite learning from a precise average case analysis of strategies for learning in the limit and illustrate our approach for the case of learning monomials. 1.
Citations
|
1364
|
A theory of the learnable
– Valiant
- 1984
|
|
624
|
Language identification in the limit
– Gold
- 1967
|
|
511
|
Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm
– Littlestone
- 1988
|
|
144
|
Learning Boolean formulae
– Kearns, Li, et al.
- 1995
|
|
59
|
Learning simple concepts under simple distributions
– Li, Vitányi
- 1991
|
|
38
|
Learnability by fixed distributions
– Benedek, Itai
- 1988
|
|
37
|
On the prediction of general recursive functions
– Barzdin, Freivald
- 1972
|
|
28
|
On the complexity of inductive inference
– Daley, Smith
- 1986
|
|
28
|
Incremental concept learning for bounded data mining
– Case, Jain, et al.
- 1999
|
|
27
|
Bias, version spaces and valiant's learning framework
– Haussler
- 1987
|
|
21
|
Incremental learning from positive data
– Lange, Zeugmann
- 1996
|
|
20
|
Set-driven and rearrangement-independent learning of recursive languages
– Lange, Zeugmann
- 1996
|
|
17
|
Average case analysis of conjunctive learning algorithms
– Pazzani, Sarrett
- 1992
|
|
15
|
PAC learning under helpful distributions
– Denis, Gilleron
- 1997
|
|
7
|
Learning one-variable pattern languages in linear average time
– Reischuk, Zeugmann
- 1998
|
|
5
|
Theoretical analysis of the nearest neighbor classifier in noisy domains
– Okamoto, Yugami
- 1996
|
|
4
|
On learning Boolean formula
– Natarajan
- 1987
|
|
4
|
An average-case analysis of k -nearest neighbor classifier
– Okamoto, Satoh
- 1995
|
|
2
|
Learning k -variable pattern languages efficiently stochastically finite on average from positive data
– Rossmanith, Zeugmann
- 1998
|
|
2
|
Proper learning algorithms for functions of k terms under smooth distributions
– Sakai, Takimoto, et al.
- 1995
|
|
2
|
Learning k-variable pattern languages efficiently stochastically finite on average from positive data
– Rossmanith, Zeugmann
- 1998
|
