Aslam, Javed and Scott Decatur. "Improved Noise-Tolerant Learning and Generalized Statistical Queries." Harvard University Technical Report TR-17-94, Center for Research in Computing Technology, Division of Applied Sciences. Lecture 20: November 23, 1994 20-11  Home/Search   Document Details and Download   Summary   Related Articles   Check

....in the case of zero noise and describe how the Perceptron Algorithm can be modi ed and combined with the procedure from the Outlier Removal Lemma to produce a polynomial time PAC learning algorithm. Finally, we describe how the algorithm can be adjusted to the noisy case using known techniques [Byl94, Kea93, AD94]. 1.2 Notation, de nitions, and preliminaries In this paper, we consider the problem of learning linear threshold functions in the PAC model in the presence of random classi cation noise [KV94] The problem can be stated as follows. We are given access to examples (points) drawn from some ....

....with Noise We now describe how the Modi ed Perceptron Algorithm can be converted to one that is robust to random classi cation noise. We present two ways of doing this. The rst is to recast the algorithm in the Statistical Query (SQ) model of Kearns [Kea93] as extended by Aslam and Decatur [AD94], and to use the fact that any SQ algorithm can be made tolerant of random classi cation noise. The second is a direct argument along the lines of Bylander [Byl94] who describes how the standard Perceptron Algorithm can be modi ed to work in this noise model. We begin with some observations ....

[Article contains additional citation context not shown here]

J. A. Aslam and S. E. Decatur. Improved noise-tolerant learning and generalized statistical queries. Technical Report TR-17-94, Harvard University, July 1994.

....with Noise We now describe how the Modified Perceptron Algorithm can be converted to one that is robust to random classification noise. We present two ways of doing this. The first is to recast the algorithm in the Statistical Query (SQ) model of Kearns [Kea93] as extended by Aslam and Decatur [AD94], and to use the fact that any SQ algorithm can be made tolerant of random classification noise. The second is a direct argument along the lines of Bylander [Byl94] who describes how the standard Perceptron Algorithm can be modified to work in this noise model. We begin with some observations ....

....is clear that given access to non noisy data, this expectation can be estimated to any desired accuracy with any desired confidence 1 Gamma ffi in time poly( 1 ; log( 1 ffi ) by simply calculating the expectation over a sufficiently large sample. Kearns [Kea93] and Aslam and Decatur [AD94] prove that one can similarly perform such an estimation even in the presence of random classification noise. 4 Specifically, for any noise rate j 1=2 and any accuracy (or tolerance) parameter , the desired expectation can be estimated with confidence 1 Gamma ffi in time (and sample size) ....

[Article contains additional citation context not shown here]

J. A. Aslam and S. E. Decatur. Improved noise-tolerant learning and generalized statistical queries. Technical Report TR-17-94, Harvard University, July 1994.

....A we will prove some of the specific bounds on sample size. Our presentation generally follows Kearns and Vazirani s textbook [14] although we provide explicit, and slightly different, bounds on sample size instead of big O bounds. Another proof of Theorem 1, below, is given by Aslam and Decatur [2]. Let us assume that we are trying to learn target concept C on instance space X , with distribution D. A statistical query learning algorithm has access to the Stat oracle instead of the usual Examples oracle. The Stat oracle will tell us statistical properties of labeled examples, instead of ....

....with tolerances of the same order of magnitude as Algorithm Forest. However, it is an interesting feature of statistical query learning algorithms that this improvement in query complexity does not lead to an improvement in sample size for the following reason (also observed by Aslam and Decatur [2]) The queries of Algorithm Forest are non adaptive in the sense that they can all be asked at the same time, without depending on the results of the other queries. Thus in this case one can use Hoeffding s Inequality (described in Appendix A) to evaluate each query simultaneously from a single ....

J. A. Aslam and S. E. Decatur. Improved noise-tolerant learning and generalized statistical queries. Technical Report TR-17-94, Center for Research in Computing Technology, Division of Applied Sciences, Harvard University, 1994.

....in the case of zero noise and describe how the Perceptron Algorithm can be modified and combined with the procedure from the Outlier Removal Lemma to produce a polynomial time PAC learning algorithm. Finally, we describe how the algorithm can be adjusted to the noisy case using known techniques [Byl94, Kea93, AD94]. 1.2 Notation, definitions, and preliminaries In this paper, we consider the problem of learning linear threshold functions in the PAC model in the presence of random classification noise [KV94] The problem can be stated as follows. We are given access to examples (points) drawn from some ....

....with Noise We now describe how the Modified Perceptron Algorithm can be converted to one that is robust to random classification noise. We present two ways of doing this. The first is to recast the algorithm in the Statistical Query (SQ) model of Kearns [Kea93] as extended by Aslam and Decatur [AD94], and to use the fact that any SQ algorithm can be made tolerant of random classification noise. The second is a direct argument along the lines of Bylander [Byl94] who describes how the standard Perceptron Algorithm can be modified to work in this noise model. We begin with some observations ....

[Article contains additional citation context not shown here]

J. A. Aslam and S. E. Decatur. Improved noise-tolerant learning and generalized statistical queries. Technical Report TR-17-94, Harvard University, July 1994.

....we desire. Lecture 20: November 23, 1994 20 7 20.4.2 Producing noise free statistics from a sample oracle with classification noise First, note that the treatment of this topic provided in [3] has some significant errors. We now present an alternate treatment, which is covered in more detail in [2]. Suppose we have a query ( to the statistics oracle. We want to return the value P r EX(c;D) 1] that is, a statistic taken with respect to the noise free oracle EX(c;D) but we have access only to a source of labelled examples with classification noise EX j CN (c; D) For the purposes ....

Aslam, Javed and Scott Decatur. "Improved Noise-Tolerant Learning and Generalized Statistical Queries." Harvard University Technical Report TR-17-94, Center for Research in Computing Technology, Division of Applied Sciences. Lecture 20: November 23, 1994 20-11

....and Engineering graduate fellowship from the Department of Defense. viii Bibliographical Notes Most of the results of this thesis have been published previously. The results contained in Chapters 3 and 5, as well as some of Chapter 6, appear in a Harvard University Technical Report HUTR 17 94 (Aslam and Decatur, 1994) and an extended abstract in COLT 95 (Aslam and Decatur, 1995) The remainder of Chapter 6 and all of Chapter 8 and Appendix B appear as an extended abstract in COLT 93 (Decatur, 1993) An extended abstract of the results in Chapter 4 and Appendix A appear in FOCS 93 (Aslam and Decatur, 1993) ....

Aslam, Javed and Scott Decatur. (1994). Improved noise-tolerant learning and generalized statistical queries. Technical Report TR-17-94, Harvard University, July.

....0 and analogously define Theta; o; When discussing asymptotics related to j, we consider j 1=2, or correspondingly (1=2 Gamma j) Gamma1 1. Learning in Hybrid Noise Environments Using Statistical Queries 263 Proof: By generalizing a technique of Laird [Lai88] Aslam and Decatur [AD94] effectively show that two important parameters determine the number of labelled examples sufficient to perform hypothesis testing by any type of example oracle. These parameters are (1) t the probability of drawing a labelled example on which the two hypotheses disagree; and (2) ff the ....

....used to estimate P in the presence of classification noise. The additional accuracy is required to accommodate error introduced by the malicious error examples. The quantities we use to estimate P are based on the classification noise alone simulation of SQ algorithms of Aslam and Decatur [AD94]. For each query [ we wish to estimate to within Sigma the value P , the probability that a labelled example drawn from EX (f; D) satisfies . Let P r be the probability that a labelled example drawn from EX j;fi CAM (f; D) satisfies given that the labelled example was not chosen ....

[Article contains additional citation context not shown here]

Javed Aslam and Scott Decatur. Improved noise-tolerant learning and generalized statistical queries. Technical Report TR-17-94, Harvard University, July 1994.

....which when polled, draws an example x according to D, labels it according to f , and then flips each component of x with probability and flips the label with probability j. We combat such an example oracle by undoing both types of noise separately. We combine techniques of Aslam and Decatur [1] with those used in the proof of Theorem 1. If we define (x; l) x; l) and let P j; be the probability that a labelled example drawn from EX j; CAT (f; D) satisfies , then P j; 1 Gamma j)P jP P j; 1 Gamma j)P jP and therefore by solving the above ....

Javed Aslam and Scott Decatur. Improved noisetolerant learning and generalized statistical queries. Technical Report TR-17-94, Harvard University, July 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC