A. Blum, A. Kalai, and H. Wasserman. Noise-Tolerant Learning, the Parity Problem, and the Statistical Query Model. Journal of the ACM, 50(4):506--519, July 2003.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....be approximated to within n 3 ffl in probabilistic time 2 fln( 1 2 1 ffl ) Proof. Our algorithm uses Ajtai s [1] reduction of SVP Length to the problem of finding a short vector in a special class of lattices; we solve the latter problem by adapting an idea of Blum, Kalai, and Wasserman [2]. To obtain the best approximation factors, we use the sharpest form of the reduction, due to Cai and Nerurkar [4, 3] For integers n; m; and q, Ajtai [1] defines a family of lattices in Z m defined by (n; m; q) fL(A)g, where A is an n Theta m matrix over Z q , and L(A) fx 2 Z m j Ax j ....

....how to express any vector u 2 Z n q as a sum of at most n ffl vectors from S; the coefficient vector clearly has Euclidean length n ffl=2 . A non zero vector in L(A) is obtained by considering the case u = 0. The arguments below are adaptations of the arguments by Blum, Kalai, and Wasserman [2]. Divide the n coordinates into a groups of b coordinates each. Recall that a = ffl log n and b = n=a. Number the groups 1 through a. We will create a 1 sample sets S 0 ; S 1 ; S a Z n q such that the following properties hold: 1) for every i, 0 i a, every v 2 S i agrees with u ....

A. Blum, A. Kalai, and H. Wasserman. Noise-tolerant learning, the parity problem, and the statistical query model. Proc. 32nd STOC , pages 435--440, 2000.

....polynomial in the input size, while still no SQ learning algorithms can learn the class efficiently. This gives an example of PAC learnable, but not SQ learnable class of functions. Previously, both [K98] and [BFJ 94] proved that the class of parity functions fits into this category, and later [BKW00] proved that a class of noisy parity functions also fits. Our example is the first class of functions in this category that are not parity functions. This result provides some insights towards better understanding of SQ learning algorithms. The rest of the paper is organizes as follow: section 2 ....

....is high, it outputs f ( X) Gamma1 , otherwise it outputs f ( X) 1 . The name of the algorithm comes from the fact that the algorithm estimates the probability by building a complete binary tree from the samples it draws. Our algorithm is inspired by the algorithm Blum et al. used in [BKW00] to learn noisy parity functions, where the main idea is also trying to write an input as logarithmically many samples. Now we describe BUILD TREE in more detail: The algorithm BUILD TREE has a random example oracle EXf , which, at each invocation, produces a random pair ( x; f a ( x) where x ....

[Article contains additional citation context not shown here]

Avrim Blum, Adam Kalai and Hal Wasserman, Noise-tolerant Learning, the Parity problem, and the Statistical Query model. In Proceedings of the 32nd Annual ACM Symposium on Theory of Computing, pp. 435--440, 2000.

....of the suggested method with a few learning problems of different nature. Of particular interest is the problem of learning in the restricted class of parity functions, where only k out of n bits are active. We show that in the MQ model we can outperforme the recent result by Blum et al. [3] and handle k = O (n Gamma c log (n) log log (n) This also provides a sharp separation between our method and the SQ model. The suggested procedure works not only for classification problems but for regression problems as well. To this end, we present a uniform upper bound on the ....

....uses statistics, as opposed to labeled instances, in the course of learning. SQ algorithms can in turn be converted to noise tolerant PAC algorithms. However, it was recently shown that there are classes which are not SQ learnable but they are still PAC learnable in the presence of labeling noise [3]. Our main result forms a link between the ability to learn in the presence of persistent noise 1 and the complexity (both sample and computational) and density (cf. section 3) of the dual learning problem: We show that these two properties provide the means to apply noise robustness to the ....

[Article contains additional citation context not shown here]

A. Blum, A. Kalai, and H. Wasserman. Noise-tolerant learning, the parity problem, and the statistical query model. In Proc. of the 22'nd Ann. ACM Symp. on Theory of Computing, 2000.

....by the National Science Foundation under Grants No. CCR 9800029 and CCR9877079. The SQ approach to developing noise tolerant algorithms was surprisingly successful, so much so that Kearns asked whether or not the SQ and PAC noise models might be equivalent [Kea93] Blum, Kalai, and Wasserman [BKW00] have very recently shown that there is a class that is efficiently learnable with noise but not efficiently SQ learnable. However, they only show that this class can be learned efficiently when the noise rate is constant. This leaves open the question of whether or not there is an efficient SQ ....

....O(log n) bits of an n bit input space can be learned (inverse polynomial) noisetolerantly more efficiently with a custom PAC algorithm than with any SQ based algorithm for this class. We actually present several somewhat different approaches to this result. First, while the Blum et al. results [BKW00] focus on producing a superpolynomial separation between PAC noise and SQ learning in the constant noise setting, they have in fact developed a family of parameterized algorithms that can be used to derive a variety of learnability results. In particular, some members of this family of algorithms ....

[Article contains additional citation context not shown here]

Avrim Blum, Adam Kalai, and Hal Wasserman. Noise-tolerant learning, the parity problem, and the Statistical Query model. In Proceedings of the Thirty-Second Annual ACM Symposium on Theory of Computing, 2000. To appear.

No context found.

A. Blum, A. Kalai, H. Wasserman, \Noise-Tolerant Learning, the Parity Problem, and the Statistical Query Model," STOC 2000 (Proc. 32nd Symp. Theory of Computing), ACM Press, 2000.

....initial instance, the error rate z t is correct, by the following lemma (due to Blum, Kalai, and Wasserman) Lemma 3 Let (al,b) as, bs) be samples from (A, b, r ) then b . bs is the correct label (1 2r ) s. for al . as with probability The proof follows by induction on s [2]. The resulting instance is distributed uniformly; thus solves it in time poly(n, log(1 e(zlt) But note that: e(l t) 1 2r ) 1 2( e(r ) 24v) so that poly(n, log(1 e(z t) poly(n, n log(1 e(z ) poly(n, log(1 e(z ) thus .4 solves adversarial instances in time ....

Avrim Blum, Adam Kalai, and Hal Wasserman. Noise-tolerant learning, the parity problem, and the statistical query model. In Proceedings of the Thirty-Second Annual ACM Symposium on Theory of Computing, Portland, Oregon, 21-23 May 2000.

.... used related ideas to reduce the space complexity of a birthday step in point counting algorithms for elliptic curves [19] Blum, Kalai, and Wasserman previously have independently discovered something closely related to the k tree algorithm for xor in the context of their work on learning theory [5]. In particular, they use the existence of a subexponential algorithm for the k sum problem when k is unrestricted to nd the rst known subexponential algorithm for the learning parity with noise problem. We note that any improvement in the k tree algorithm would immediately lead to improved ....

A. Blum, A. Kalai, H. Wasserman, \Noise-Tolerant Learning, the Parity Problem, and the Statistical Query Model," STOC 2000 , ACM Press, 2000.

....correct, by the following lemma (due to Blum, Kalai, and Wasserman) Lemma 3 Let (a 1 ; b 1 ) a s ; b s ) be samples from (A; b; j) then b 1 : b s is the correct label for a 1 : a s with probability 1 2 1 2 (1 Gamma 2j) s . The proof follows by induction on s [2]. The resulting instance is distributed uniformly; thus A solves it in time poly(n; log(1=ffl(j 0 ) But note that: ffl(j 0 ) 1 2 (1 Gamma 2j) n 1 = 1 2 (1 Gamma 2( 1 2 Gamma ffl(j) n 1 = 1 2 (2ffl(j) n 1 so that poly(n; log(1=ffl(j 0 ) poly(n; n ....

Avrim Blum, Adam Kalai, and Hal Wasserman. Noise-tolerant learning, the parity problem, and the statistical query model. In Proceedings of the Thirty-Second Annual ACM Symposium on Theory of Computing, Portland, Oregon, 21--23 May 2000.

No context found.

A. Blum, A. Kalai, and H. Wasserman. Noise-Tolerant Learning, the Parity Problem, and the Statistical Query Model. Journal of the ACM, 50(4):506--519, July 2003.

No context found.

A. Blum, A. Kalai, and H. Wasserman. Noise-Tolerant Learning, the Parity Problem, and the Statistical Query Model. J. ACM 50(4): 506--519, 2003.

No context found.

A. Blum, A. Kalai, and H. Wasserman. Noise-Tolerant Learning, the Parity Problem, and the Statistical Query Model. Journal of the ACM, 50(4):506--519, July 2003.

No context found.

A. Blum, A. Kalai, and H. Wasserman. Noise-tolerant learning, the parity problem, and the statistical query model. In Proceedings of the 32nd Annual ACM Symposium on Theory of Computing, 2000. To appear.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC