J.A. Aslam and S.E. Decatur. Specification and Simulation of Statistical Query Algorithms for E#ciency and Noice Tolerance. Journal of Computer and System Sciences, 56(2):191-208, 1998.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....is that it is very sensitive to the labels of individual examples. So in Chapter 4 we take a different approach. We present an algorithm that, rather than examining the labels of individual examples, asks questions relating to statistics about the examples. These statistical query (SQ) algorithms [52, 8, 9, 22] can also be shown to meet the PAC criteria. This holds even in the presence of classification noise, in which each example (with some probability 1=2) has its label changed before being given to the learner. SQ algorithms are also known to tolerate other types of noise. In Chapter 5 we change ....

....j (the classification noise rate) X s label is flipped before given to the learner. We define D j to be this noisy distribution. To obtain a noise tolerant version of the algorithm of Chapter 3 we use the statistical query model, introduced by Kearns [52] and enhanced by Aslam and Decatur [8, 9] and Decatur [22] In this model, rather than sampling labeled examples, the learner requests the value of various statistics. An additive statistical query 6 If instead the VC dimension is bounded by p(k; n)m ff for some 0 ff 1, then a different result from Blumer et al. applies. 19 (SQ) ....

[Article contains additional citation context not shown here]

J. A. Aslam and S. E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. Journal of Computer and System Sciences, 1998. To appear. Earlier version in COLT '95. 171

.... Malicious Noise The learning model we consider is essentially the malicious noise extension of Valiant s widely studied Probably Approximately Correct (PAC) model [34] This noise model was introduced by Valiant [35] and has been studied extensively by Kearns and Li [19] and other researchers [2, 3, 4, 11, 13, 26]. Let EX j MAL ( u; D) be a malicious example oracle that behaves as follows when invoked: with probability 1 Gamma j it returns a clean example which is a pair h x; sign( u Delta x)i where 2 x is drawn from the distribution D over B(R) With probability j; though, EX j MAL ( u; D) ....

....a hypothesis which accurately approximates the underlying linear threshold function in the absense of noise. The sample complexity of a learning algorithm in this model is the number of times it queries the example oracle. A slightly stronger model of malicious noise has also been proposed [2, 11]. In this model first a clean sample of the desired size is drawn from a noise free oracle; then each point in the sample is randomly and independently selected with probability j; then the adversary replaces each of the selected points with a dirty example of its choice; and finally the corrupted ....

[Article contains additional citation context not shown here]

J. Aslam and S. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance, J. Comput. Syst. Sci. 56 (1998), 191-208.

....data corrupted by random classification noise [Kea93] Thus, any concept class learnable from statistical queries is also PAC learnable in the presence of random classification noise. There are several variants to the formulation given above that improve the efficiency of the simulation [AD93, AD98] but they are all polynomially related. 1 Normally, one would also require polynomial dependence on 1= 1=2 Gamma j) in part because normally this is easy to achieve (e.g. it is achieved by any statistical query algorithm) Our algorithms run in polynomial time for any fixed j 1=2, but ....

J. A. Aslam and S. E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. Journal of Computer and System Sciences, 56(2):191--208, April 1998.

....was little work to characterize which concept classes could be efficiently learned in the presence of noise. The first (computationally feasible) tool to design noise tolerant PAC algorithms was provided by the statistical query model, first introduced by [Kearns, 1993] and since improved by [Aslam and Decatur, 1997]. In this model, rather than sampling labeled examples, the learner requests the value of various statistics. A relative error statistical query [Aslam and Decatur, 1997] takes the form SQ( where is a predicate over labeled examples, is a relative error bound, and is a threshold. As an ....

....PAC algorithms was provided by the statistical query model, first introduced by [Kearns, 1993] and since improved by [Aslam and Decatur, 1997] In this model, rather than sampling labeled examples, the learner requests the value of various statistics. A relative error statistical query [Aslam and Decatur, 1997] takes the form SQ( where is a predicate over labeled examples, is a relative error bound, and is a threshold. As an example, let to be (h(x) 1) 0) 15 which is true when x is a negative example but the hypothesis classifies x as positive. So the probability that is true for ....

[Article contains additional citation context not shown here]

Aslam, J. A. and Decatur, S. E. 1997. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. Journal of Computer and System Sciences . To appear. 36

.... an SQ oracle for C for any function f 2 C, produces a function h : f0; 1g n f0; 1g such that Pr x Un [f(x) 6= h(x) In this paper we consider only the original additive error version of statistical queries and not the relative error model, which is in some sense polynomially equivalent [AD98] These definitions can be generalized to arbitrary probability distributions rather than Un in an obvious way. However, in this paper, our focus is on the uniform distribution. In the sequel, probabilities and expectations that do not specify a distribution are over the uniform distribution on ....

Javed A. Aslam and Scott E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. Journal of Computer and System Sciences, 56(2):191--208, April 1998.

....evidence that the class of one dimensional patterns is significantly more complex than the union of intervals on the real line, observe that the consistency problem for the latter class is trivial to solve. Goldman and Scott [13] gave an efficient algorithm that uses the statistical query model [2, 18] to PAC learn the class of continuous one dimensional geometric patterns under high noise rates (any rate 1=2 of classification noise) They also performed an empirical study of how well their algorithm worked on both simulated and real data. In Section 5 we give an on line, agnostic algorithm ....

J. A. Aslam and S. E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. Journal of Computer and System Sciences, 1998. To appear.

....correct label. Thus the example drawn is labeled incorrectly, at random, with probability j. In the malicious noise model [32] with probability j the adversary can provide an example and label of its choice. To obtain a noise tolerant version of our algorithm we use the statistical query model [28, 20, 4, 5]. In this model, rather than sampling labeled examples, the learner requests the value of various statistics on the distribution from an oracle. A statistical query oracle returns the probability, within some additive constant, that some predicate is true relative to the distribution. The ....

....learner requests the value of various statistics on the distribution from an oracle. A statistical query oracle returns the probability, within some additive constant, that some predicate is true relative to the distribution. The particular queries we use are known as relative statistical queries [5]. These take the form rel statD( where is a predicate over labeled examples drawn from D, is the relative error bound, and is the threshold. For target function f , let P = PrD [ x; f(x) 1] If P then rel statD( may return . If is not returned, then rel statD( ....

[Article contains additional citation context not shown here]

Javed A. Aslam and Scott E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. In Proceedings of the Eighth Annual ACM Conference on Computational Learning Theory, pages 437--446, July 1995.

....NP complete to find some one dimensional geometric pattern (of any number of points) that correctly classifies all examples in S. By contrast, for the union of intervals the consistency problem is trivial. Goldman and Scott [7, 8] gave an efficient algorithm that uses the statistical query model [2, 12] to PAClearn the class of continuous one dimensional geometric 4 patterns under high noise rates (any rate 1=2 of classification noise) They also performed an empirical study of how well their algorithm worked on both simulated and real data. 4 A GENERAL LEARNING ALGORITHM FOR GEOMETRIC ....

J. Aslam and S. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. In Proc. 8th Annu. Conf. on Comput. Learning Theory, pages 437--446. ACM Press, New York, NY, 1995.

.... result of this paper is a polynomial time algorithm that PAC learns the class of one dimensional geometric patterns under high noise rates (any rate 1=2 of classification noise) We obtain such a noise tolerant algorithm by using the recently developed statistical query model (Kearns (1993) Aslam and Decatur (1997)) A second contribution of this work is an empirical study of how an algorithm designed using the statistical query model works on simulated and real data. One goal of this empirical work was to contrast the empirically determined training set sizes with the worst case lower bounds from our ....

....j (the classification noise rate) X s label is flipped before given to the learner. We define D j to be this noisy distribution. A NOISE TOLERANT PATTERN LEARNING ALGORITHM 9 To obtain a noise tolerant version of our algorithm we use the statistical query model (Aslam and Decatur (1993) Aslam and Decatur (1997), Decatur (1993) Kearns (1993) In this model, rather than sampling labeled examples, the learner requests the value of various statistics. An additive statistical query (SQ) oracle returns the probability, within some additive constant, that a provided predicate is true for a random labeled ....

[Article contains additional citation context not shown here]

Aslam, A., & Decatur, S. (1997). Specification and simulation of statistical query algorithms for efficiency and noise tolerance. Journal of Computer and System Sciences. To appear.

.... A key result of this paper is an efficient algorithm that PAC learns the class of one dimensional geometric patterns under high noise rates (any rate 1=2 of classification noise) We obtain such a noise tolerant algorithm by using the recently developed statistical query model (Kearns 1993, Aslam and Decatur 1995). A second contribution of this work is an empirical study of how well an algorithm designed using the statistical query model works on simulated data. One goal of this study was to contrast the empirically determined training set sizes with the worst case bounds from our theoretical analysis. ....

....any concept C 2 C consistent with a sample of size 3 max Gamma 4 ffl lg 2 ffi ; 8d ffl lg 13 ffl Delta has error at most ffl with probability at least 1 Gamma ffi . To obtain a noise tolerant version of our algorithm we use the statistical query model (Aslam and Decatur 1993, Aslam and Decatur 1995, Decatur 1993, Kearns 1993) In this model, rather than sampling labeled examples, the learner requests the value of various statistics. A relative statistical query (Aslam and Decatur 1995) takes the form SQ D ( where is a predicate over labeled examples, is a relative error bound , ....

[Article contains additional citation context not shown here]

J. Aslam & S. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. In Proc. of the 8th Ann. ACM Conf. on Computational Learning Theory, pp 437--446, 1995.

....Defense. viii Bibliographical Notes Most of the results of this thesis have been published previously. The results contained in Chapters 3 and 5, as well as some of Chapter 6, appear in a Harvard University Technical Report HUTR 17 94 (Aslam and Decatur, 1994) and an extended abstract in COLT 95 (Aslam and Decatur, 1995). The remainder of Chapter 6 and all of Chapter 8 and Appendix B appear as an extended abstract in COLT 93 (Decatur, 1993) An extended abstract of the results in Chapter 4 and Appendix A appear in FOCS 93 (Aslam and Decatur, 1993) Most of the results of Chapter 9 appear as an extended abstract ....

Aslam, Javed and Scott Decatur. (1995). Specification and simulation of statistical query algorithms for efficiency and noise tolerance. In Proceedings of the Eighth Annual ACM Workshop on Computational Learning Theory. ACM Press, July.

No context found.

J.A. Aslam and S.E. Decatur. Specification and Simulation of Statistical Query Algorithms for E#ciency and Noice Tolerance. Journal of Computer and System Sciences, 56(2):191-208, 1998.

No context found.

Javed A. Aslam and Scott E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. In Proceedings of the Eighth Annual Workshop on Computational Learning Theory, pages 437--446, 1995.

No context found.

J.A. Aslam and S.E. Decatur. Specification and Simulation of Statistical Query Algorithms for E#ciency and Noice Tolerance. Journal of Computer and System Sciences, 56(2):191-208, 1998.

No context found.

J. A. Aslam and S. E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. In Proceedings of the Eighth Annual ACM Conference on Computational Learning Theory, pages 437--446, July 1995.

No context found.

J. A. Aslam and S. E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. J. Comput. Syst. Sci., 56(2):191--208, April 1998.

No context found.

J. A. Aslam and S. E. Decatur. Specification and simulation of statistical query algorithms for efficiency and noise tolerance. Journal of Computer and System Sciences, 1998. To appear. Earlier version in COLT '95. 171

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC