D. Haussler, M. J. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):129--161, 1991.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....(i) of Section 1.4, their sample complexity bound is O(K ##) Depending on the desired confidence #, their sample complexity bound of O(K ##) can be greater than or less than the bound of O( 1 #) K ln 1 # ln 1 #) that we use. The bounds may be reconciled by the opportunity to boost confidence [8]. In particular, loosely speaking, in the design of learning algorithms a factor of ln 1 # can be traded for a multiple of 1 #. Interestingly, this suggests that the bound of [3] which ensures near feasibility of a single optimal solution to the RLP is equivalent up to a constant factor to ....

D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):129--161, 1991.

....Em (f ; x)j =2g; it follows that K S. Hence q(m; 2; Q) Q (K) 2 = q(m; P ) 2 =2: Thus the function q has property C. 4 PAC Learnability In this section it is shown that if a pair (F ; P) is PAC learnable, then so is (F ; P) We begin with the following result ([3], or [8] Theorem 3.1) Lemma 2 Given X;S;P;F , suppose there exists a family of maps fBm g m 1 where Bm : 0; 1) X [0; 1] F with the following property: For each ; 0 there exists an integer s = s( such that : d P [f; h s (f ; x) g ; 4) 5 where h s (f ; x) ....

D. Haussler, M. Kearns, N. Littlestone and M. K. Warmuth, Equivalence of models for polynomial learnability, Information and Computation, 95, 1991, 129-161. 11

....4.1.3 Remarks ffl Typically L will be polynomial in ln 1 ffi , not just 1 ffi . ffl Defining PAC learning with two oracles, one for positive examples and one for negative examples, results in an equivalent model. See problem 1.3 in the Kearns Vazirani text. The solution may be found in [2]. ffl This is a worst case model: L must meet its accuracy and confidence requirements even for the worst possible choices of D and c. As a result, many of the results about PAC learning in the literature are negative results, showing that a certain class C is not PAC learnable. ffl As defined, ....

D. Haussler, M. Kearns, N. Littlestone, and M.K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):129-161, 1991.

....ones are deterministic) Since a learning algorithm is allowed, by definition, to fail with some low probability (over the sample space) it seems natural to let it use random bits, and allow for some failure probability over these random bits. Furthermore, the derandomization schemes shown in [14] can be used to transform the randomized algorithms into deterministic algorithms, while preserving efficiency and noise tolerance. Efficient O(log n) wRFA learnability can serve also as sufficient condition for efficient noise tolerant PAC learnability. However, in this case we have to restrict ....

David Haussler, Michael Kearns, Nick Littlestone, and Manfred K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95:129-- 161, 1991.

.... fl in poly(1=fl) time. Define OE : 0; 1] 0; c 1 (fi) by OE(x) c 1 (fi)x c2 (fi) Consider the Algorithm B that uses as guesses for b all values of OE Gamma1 (z) for multiples z of ffl=4, sets fl = ffl=4, calls Algorithm A for each of these values, then uses hypothesis testing as in [7] to estimate which of those roughly 4=ffl hypotheses is the best. One of the poly(1 ffl) runs would produce a hypothesis with error at most c 1 (fi)er P (F ) c2 (fi) ffl=2, and hypothesis testing can be applied to find from among a set of hypothesis with one such good one a hypothesis with ....

D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95:129--161, 1991.

....learnability is in fact equivalent to PAC learnability where the class priors are concealed from the learner. Formally, this is the equivalence of the standard PAC framework with the two button version, where the learner has access to a positive example oracle and a negative example oracle [14]. The two button version conceals the class priors and only gives the learner access to the distribution as restricted to each class label. We assume that neither unsupervised learner knows whether it is receiving the positive or the negative examples. Consequently both learners must apply the ....

D. Haussler, M. Kearns, N. Littlestone and M.K. Warmuth (1991). Equivalence of Models for Polynomial Learnability. Information and Computation, 95(2), pp. 129161.

....probability weight of the instance space. Thus it is not hard to show that these concept classes are trivially PAC learnable. One goal of our research is to build a framework for studying such problems. To study learning algorithms for these concept classes we extend the basic mistake bound model [13, 14, 18] to the cases that a helpful teacher or the learner selects the query sequence, in addition to the cases where instances are chosen by an adversary or according to a probability distribution on the instance space. Previously, helpful teachers have been used to provide counterexamples to ....

David Haussler, Michael Kearns, Nick Littlestone, and Manfred K. Warmuth. Equivalence of models for polynomial learnability. In Proceedings of the 1988 Workshop on Computational Learning Theory, pages 42--55. Morgan Kaufmann, August 1988.

....complexity depends exclusively on the VC dimension of the target concept class and the error and confidence parameters and ffi, respectively. This model has been generalized by allowing the sample size to depend on the concept complexity, too (cf. e.g. Blumer et al. 3] and Haussler et al. [10]) Provided no upper bound for the concept complexity of the target concept is given, such PAC learners decide themselves how many examples they wish to read (cf. 10] This feature is also adopted to our setting of stochastic finite learning. However, all variants of PAC learning we are aware of ....

.... generalized by allowing the sample size to depend on the concept complexity, too (cf. e.g. Blumer et al. 3] and Haussler et al. 10] Provided no upper bound for the concept complexity of the target concept is given, such PAC learners decide themselves how many examples they wish to read (cf. [10]) This feature is also adopted to our setting of stochastic finite learning. However, all variants of PAC learning we are aware of require that all hypotheses from the relevant hypothesis space are uniformly polynomially evaluable. Though this requirement may be necessary in some cases to achieve ....

Haussler, D., Kearns, M., Littlestone, N., & Warmuth, M.K. (1991). Equivalence of models for polynomial learnability. Information and Computation, 95, 129--161.

.... c 1 ( b c2 ( in poly(1= time. De ne : 0; 1] 0; c 1 ( by (x) c 1 ( x c2 ( Consider the Algorithm B that uses as guesses for b all values of 1 (z) for multiples z of =4, sets = 4, calls Algorithm A for each of these values, then uses hypothesis testing as in [7] to estimate which of those roughly 4= hypotheses is the best. One of the poly(1 ) runs would produce a hypothesis with error at most c 1 ( er P (F ) c2 ( 2, and hypothesis testing can be applied to nd from among a set of hypothesis with one such good one a hypothesis with error at ....

D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95:129-161, 1991.

....there is a solution then, with probability at least 1=2, A outputs a solution. The probability that the algorithm fails in k attempts is at most (1=2) k , which approaches zero very rapidly with increasing k. Computational Complexity of Learning 33 The following result is from [83] See also [79, 51]. Theorem 8.1 Let H = S H n be a hypothesis space and suppose that there is a PAC learning algorithm for H which is efficient with respect to accuracy and example size. Then there is a randomised algorithm which solves the problem of finding a hypothesis in H n consistent with a given training ....

....ffl. This is not really so different from the basic PAC learning framework: in the original PAC model as introduced by Valiant, a learning algorithm was given the accuracy and confidence parameters and had access to an oracle generating random labeled examples. Subsequently, Haussler et al. [51] showed that this model is equivalent to the functional model we described in earlier sections, in which the learning algorithm is given only a training sample as input. Suppose that a sample s = x 1 ; y 1 ) x m ; y m ) of points from X Theta Y is given. The observed error (or ....

D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability. Inform. Comput., 95(2):129--161, December 1991.

....sample complexity, overestimating the parameters will still guarantee success with high probability. For item number 2, we must determine if a PAC algorithm successfully created a hypothesis with error at most ffl. We do this using hypothesis testing, a technique presented by Haussler et al. [43]. The goal is: given a hypothesis H, an error parameter ffl, and access to a labeled sample drawn according to distribution D, determine with high probability if H s error is at most ffl. It is not possible to distinguish a hypothesis with error ffl from one with error just greater than ffl, but ....

....error bound if it misclassifies at most 3mffl=4 of the examples. Otherwise, reject H. Haussler et al. showed that the probability H is accepted if its error is ffl is at most ffi=2 i 1 , and the probability H is rejected if its error is ffl=2 is at most ffi=2 i 1 . The doubling procedure [43] starts with i = 1 and estimates 7 k with k = j 2 (i Gamma1) ln(2=ffi) k and then runs the PAC algorithm A with error parameter ffl=2, confidence parameter 1=2, and k as the size of the target concept. It then takes the hypothesis H i produced by A and tests it. If H i is accepted, we ....

[Article contains additional citation context not shown here]

D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):129-- 161, December 1991.

....for all , pairs. De nition 3 A PAC boosting algorithm is a generic algorithm which can leverage any weak PAC learner to meet the strong PAC learning criteria. In the remainder of the paper we emphasize boosting the accuracy as it is much easier to boost the con dence , see Haussler et al. [13] and Freund [14] for details. Furthermore, we emphasize boosting by re sampling, where the strong PAC learner draws a large sample, and each iteration the weak learning algorithm is called with some distribution over this sample. 1 To simplify the presentation we omit the instance space ....

David Haussler, Michael Kearns, Nick Littlestone, and Manfred K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):129{ 161, December 1991.

....to PAC learnability where the relative frequency of positive negative examples is concealed from the learner. Formally, this is the equivalence of the standard PAC framework with the two button version, where the learner has access to a positive example oracle and a negative example oracle [10]. The two button version conceals the class priors and only gives the learner access to the distribution as restricted to each class label. We assume that neither unsupervised learner knows whether it is receiving the positive or the negative examples. For a concept class that is closed under ....

D. Haussler, M. Kearns, N. Littlestone and M.K. Warmuth (1991). Equivalence of Models for Polynomial Learnability. Information and Computation, 95(2), pp. 129161.

....that has been drawn. regfinkl.tex; 7 01 2000; 13:33; p. 10 Learning regular languages from simple positive examples 11 The parameter l can be omitted from the input parameters and guessed by the learning algorithm since the learner can test whether a hypothesis is good enough for the learning task (Haussler et al. 1991). Previous de nition can be relaxed in the following way: the output of the learning algorithm can belong to a larger hypothesis class. We say that a class of language is predictable if it is PAC learnable in some polynomially evaluatable hypothesis class. The main result concerning the PAC ....

....: 2 jrj ) the size of the longest example seen cannot appear among the parameters kept for measuring the time complexity of a learning algorithm. Otherwise, retaining the examples seen would always be a sucient learning strategy But note also that the hypothesis testing procedure designed in (Haussler et al. 1991) cannot be used regfinkl.tex; 7 01 2000; 13:33; p.12 Learning regular languages from simple positive examples 13 anymore since too long examples cannot be handled. Therefore, it seems impossible to omit the input parameter l. The Kolmogorov complexity of a string depends on the reference Turing ....

[Article contains additional citation context not shown here]

Haussler, D., Kearns, M., Littlestone, N., and Warmuth, M. K. (1991). Equivalence of models for polynomial learnability. Inform. Comput., 95(2):129-161.

....than a single distribution D, we assume the existence of two distributions, D 0 over X and D 1 over Y . The training instances are the union of an independent sample according to D 0 and an independent sample according to D 1 . This is similar to the two button learning model in classification [9]. The training set, then, consists of all pairs of training instances. Consider the movie recommendation task as an example of this model. The model suggests that movies viewed by a person can be partitioned into an independent sample of good movies and an independent sample of bad movies. This ....

David Haussler, Michael Kearns, Nick Littlestone, and Manfred K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):129--161, December 1991.

....the description language R associated with the concept class. On the other hand, an improper (or representation independent) learning algorithm may output any polynomial time algorithm as a hypothesis. The less constrained model of improper learning is equivalent to polynomial PACpredictability [HLW94, HKLW91] Throughout this paper, we are concerned mostly with improper learning, and the default interpretation of learnable should be that of improper learning. A well investigated alternative model of learning is that of exact learning from equivalence queries [Ang88] In this model, the learner ....

.... passively in this way [Ang90, KV94, PW90] Consequently, researchers have considered augmenting 1 If the demand of polynomial time computation below is replaced with expected polynomial time computation, then the learning algorithm need not be given the parameter s, but could guess it instead [HKLW91]. 6 this learning protocol by allowing the learner to perform experiments. In addition to drawing a randomly labeled example (or posing a hypothesis, in the exact model) the learner can perform membership queries, as defined previously, in which it supplies an example x 2 X and is told the ....

D. Haussler, M. Kearns, N. Littlestone, and M. Warmuth. Equivalence of models for polynomial learnability. Inform. Comput., 95(2):129--161, December 1991. 41

No context found.

D. Haussler, M. Kearns, N. Littlestone, M. Warmuth. Equivalence of models for polynomial learnability. Publishers,

No context found.

David Haussler, Michael Kearns, Nick Littlestone, and Manfred K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):129--161, December 1991.

No context found.

D. Haussler, M. J. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95(2):129--161, 1991.

No context found.

D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth, Equivalence of models for polynomial learnability, Proceedings of the 1988.

No context found.

Haussler, D., Kearns, M., Littlestone, N., & Warmuth, M.K. (1991). Equivalence of models for polynomial learnability. Information and Computation, 95, 129-161.

No context found.

D. Haussler, M.J. Kearns, N. Littlestone and M.K. Warmuth. Equivalence of Models for Polynomial Learnability. Information and Computation, 95(2): 129-161, 1991.

No context found.

D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability. Inform. Comput., 95(2):129-161, December 1991.

No context found.

) Haussler, D., Kearns, M., Littlestone, N., and Warmuth, M., "Equivalence of Models for Polynomial Learnability," in Proc. of the 1st Workshop on Computational Learning Theory, pp. 34-50, 1988.

No context found.

D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth, "Equivalence of models for polynomial learnability," Inform. Computat., vol. 95, pp. 129--161, 1991.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC