Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....m would solve the problem of learning DNF. Observe that the class considered by Frazier et al. is a generalization of the class of DNF formulas in which all variables only appear negated. While there has been some work addressing the general issue of mislabelled training examples in the PAC model [1, 18, 28, 17], there has been little research on learning geometric concepts with noise. Auer [3] investigates exact learning of boxes where some of the counterexamples, given in response to equivalence queries, are noisy. Auer shows that box n is learnable using hypotheses from box n if and only if the ....

Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988.

....model replaces the (deterministic) classification of the examples supplied to the learner, by a random i.i.d. variable that has some fixed probability j of assuming the inverse of the true classification value, and with probability (1 Gamma j) assumes the correct classification value [21, 28, 2] (for a stronger model of noise see [18, 12] The task of providing methods by which efficient learning algorithms can be strengthened to noise tolerant ones is one of the most important in computational learning theory. As this task seems to be very difficult, one naturally seeks partial ....

Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1st Annual Workshop on Computational Learning Theory, pages 91--96, 1988.

....framework of computational learning theory. First, it falls within the framework of learning with persistent noise. Here one assumes that the function f is derived from some function in the class C by adding noise to it. Typical works in this direction either tolerate only small amounts of noise [2, 41, 21, 39] (i.e. that the function is modi ed only at a small fraction of all possible inputs) or assume that the noise is random [1, 26, 20, 25, 33, 13, 36] i.e. that the decision of whether or not to modify the function at any given input is made by a random process) In contrast, we take the setting ....

Robert Sloan. Types of noise in data for concept learning (extended abstract). Proceedings of the 1988 Workshop on Computational Learning Theory, pp. 91-96, MIT, ACM Press, 1988.

....the adversary s probability constraint by asking repeatedly about that example and thereby statistically determine its correct classification. This notion is known as persistent malicious misclassification noise. A number of authors have investigated this and related models (Angluin Laird, 1988; Sloan, 1988; Shackelford Volper, 1988; Auer, 1993; Decatur, 1993; Kearns Li, 1993; Ron Rubinfeld, 1993; Angluin Slonim, 1994; Angluin, 1994; Angluin Krikis, 1994; Sloan Tur an, 1994; Frazier et al. 1994) The graphs we have been manipulating admit the random construction of a number of ....

Sloan, R. (1988). Types of noise in data for concept learning. Proceedings of the 1988 Workshop on Computational Learning Theory (pp. 91--96). San Mateo, CA: Morgan Kaufmann.

....to prove that a malicious error rate of at most O(ffl) is tolerable when pac learning any distinct concept class C. A number of other papers further explore various models in which the examples themselves or their classifications are corrupted (see Laird, 1987; Shackelford and Volper, 1988; Sloan, 1988, 1989; among others) Less is known about errors in query models. Sakakibara (1991) proposes a model of noise in queries, which assumes that every time a query is asked there is some independent probability of getting the wrong answer. Sakakibara gives a general technique to repeat a query ....

Sloan, R. (1988). Types of noise in data for concept learning. In Proceedings of the 1988 Workshop on Computational Learning Theory, (pp. 91-96). Cambridge, MA: Morgan Kaufmann.

....the PAC and exact models, both with and without membership queries, assumes that examples are labeled either positive or negative. In these situations the border between the positive and negative examples is well defined. There has been work addressing the issue of mislabeled training examples [AL88, Lai88, Slo88, SV88, SS92, Kea93, KL93, GS95, RR95]. In these situations, the border between the positive and negative examples may appear blurry to the learner, but this is just the result of the noise process that has been applied to the properly labeled example. There has also been some work considering learning from noisy membership queries ....

R. Sloan. Types of noise in data for concept learning. In Proc. 1st Annu. Workshop on Comput. Learning Theory, pages 91--96, Morgan Kaufmann, San Mateo, CA, 1988.

....Query (SQ) model [78] which was originally aimed at the random classi cation noise model, and later extended to include the malicious noise model [37] In this chapter we review concept learning in a noisy environment. We start with a brief discussion of various noise models (following Sloan [108] with the necessary updates) We then move on to formalize the model of PAC learning in the presence of noise. Thereafter, we focus our attention on random classi cation noise. In Section 3.2 we present the most straightforward strategy of dealing with noise Minimal Disagreement, and we discuss ....

....made with an instance space of f0; 1g n . Having a noise free example, h(x 1 ; x n ) li, the noisy oracle adds noise to this example by independently ipping each bit x i to x i with probability . Note that the label l never changes. This model was introduced and studied by Sloan [108] . In particular, Sloan was able to show that unlike he case of classi cation noise model, minimal disagreement approach is not 32 e ective against random attribute noise (cf. Sec. 3.2) Other techniques for dealing with uniform attribute noise were used by Goldman and Sloan at [58] Similarly to ....

[Article contains additional citation context not shown here]

R. H. Sloan. Types of noise in data for concept learning. In D. Haussler and L. Pitt, editors, First Workshop on Computational Learning Theory, pages 91-96. Morgan Kaufman, 1988.

....(x; However, with probability j, the learner receives the example (x; So in this noise model, learner usually gets a correct example, but some small fraction j of the time the learner receives an example in which the label has been inverted. In the model of Malicious Classification Noise([Sloan, 1988]) with probability 1 Gamma j, the learner receives the uncorrupted example (x; However, with probability j, the learner receives the example (x; 0 ) in which x is unchanged, but the label 0 is selected by an adversary who has infinite computing power and has knowledge of the learning ....

....learner usually gets a correct example, but some small fraction j of the time the learner gets 14 noisy examples and the nature of the noise is unknown. We now define two noise models that are only defined when the instance space is f0; 1g n . In the model of Uniform Random Attribute Noise ([Sloan, 1988]) the example (b 1 Delta Delta Delta b n ; is corrupted by a random process that independently flips each bit b i to b i with probability j for 1 i n. Note that the label of the true example is never altered. In this noise model, the attributes values are subject to noise, but that ....

Sloan, R. 1988. Types of noise in data for concept learning. In Proc. 1st Annu. Workshop on Comput. Learning Theory (San Mateo, CA, 1988). Morgan Kaufmann, pp. 91--96.

....performance. We aim to provide some theoretical explanation by identifying features of an input distribution that make it helpful and give associated sample size bounds. We mention relationships with other learning frameworks. The RFA setting is more benign than the random attribute noise [24, 36] scenario. A data set with missing components can be converted to one with random attribute noise by inserting random values for the missing components (although note that for k RFA data, with small k, the associated noise rate would be quite high) Finally, observe that there is a similarity to ....

R.H. Sloan (1988). Types of noise in data for concept learning, First Workshop on Computational Learning Theory, 91-96, Morgan-Kaufman.

....insight into the degree of additional information that membership queries provide a learning algorithm. Some work has been done on errors in the distribution free model of learning introduced by Valiant [Val84] This includes studies of malicious and random errors in classification and attributes [Val85,Kea89,AL88,Lai87,SV88,Slo88,Slo89]. There has been less work on errors in query models. Sakakibara [Sak90] proposes a model of noise in queries, which assumes that every time a query is asked there is some independent probability of getting the wrong answer. Sakakibara gives a general technique to repeat a query sufficiently often ....

R. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988 Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 1988, pp. 91-96.

....sample size bounds. Note that data fusion motivates 1 RFA in particular, by comparison with k RFA for k 1, since the separate components of the observations are uncorrelated. We mention relationships with other learning frameworks. The RFA setting is more benign than the random attribute noise [18, 25] scenario. A data set with missing components can be converted to one with random attribute noise by inserting random values for the missing components (although note that for k RFA data, with small k, the associated noise rate would be quite high) Finally, observe that there is a similarity to ....

R.H. Sloan (1988). Types of noise in data for concept learning, First Workshop on Computational Learning Theory, 91-96, Morgan-Kaufman.

....discrete functions, real valued functions and vector valued functions. 2. Some practitioners are wary of the assumption that the examples are generated from an underlying target function , and are not satisfied with the noise models that have been proposed to weaken this assumption (e.g. 10] [116] [115] They would like to see more general regression models investigated in which the y component in a training example (x; y) 2 X Theta Y is randomly specified according to a conditional distribution on Y , given x. Here the general goal is to approximate this conditional distribution for ....

R. Sloan. Types of noise in data for concept learning. In Proc. 1988 Workshop on Comp. Learning Theory, pages 91--96, San Mateo, CA, 1988. Morgan Kaufmann.

....functions with better than 1 Gamma err(0) accuracy. In some sense, the proof of Theorem 1 assumes that the perceptron will make mistakes on the worst examples, i.e. all the examples misclassified by w , plus those that make the least progress towards w . For various kinds of noise (Sloan, 1988), one might be able to demonstrate that the perceptron is better behaved. More generally, whenever it is hard to learn near perfect or near optimal concepts, learning good suboptimal approximations is still a reasonable possibility. Understanding when this is possible is an important direction for ....

Sloan, R. (1988). Types of noise in data for concept learning. In Proc. First Annual Workshop on Computational Learning Theory, pages 91--96.

....noise. This extension was first examined in the learning theory literature by Angluin and Laird [1] who formalized the simplest type of white label noise and then sought algorithms tolerating the highest possible rate of noise. In addition to being the subject of a number of theoretical studies [1, 15, 24, 11], the classification noise model has become a common paradigm for experimental machine learning research. Angluin and Laird provided an algorithm for learning boolean conjunctions that tolerates a noise rate approaching the information theoretic barrier of 1=2. Subsequently, there have been some ....

....1 Gamma ffi satisfies error(h) ffl. This probability is taken over the random draws from D made by EX (f; D) and any internal randomization of L. We call ffl the accuracy parameter and ffi the confidence parameter. 3 The Classification Noise Model The well studied classification noise model [1, 15, 11, 24, 14, 20, 22] is an extension of the Valiant model intended to capture the simplest type of white noise in the labels seen by the learner. We introduce a parameter 0 j 1=2 called the noise rate, and replace the oracle EX (f; D) with the faulty oracle EX j CN (f; D) where the subscript is the acronym for ....

[Article contains additional citation context not shown here]

Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988 Workshop on Computational Learning Theory, pages 91--96, August 1988.

....is not. Several results have been obtained for learning in the presence of errors in the Probably Approximately Correct model introduced by Valiant [38] These include results for learning in the presence of malicious and random noise in classification and attributes [39] 3] 25] 22] 20] [34], 35] 37] 36] In recent work Kearns [21] identifies and formalizes a sufficient condition on learning algorithms in Valiant s model that permits the immediate derivation of noise tolerant learning algorithms. He introduces a new model of learning from statistical queries and shows that any ....

R. H. Sloan. (1988). Types of noise in data for concept learning. Proceedings of the 1988 Workshop on Computational Learning Theory (pp. 91--96). Santa-Cruz, CA: The Association for Computing Machinery.

....of formulas; these are fixed sequences of instances for which every unique formula in the class induces a different labeling. We also prove that our algorithms are robust against a large amount of random misclassification noise, similar to, but slightly more general than that considered by Sloan [23] and Angluin and Laird [2] Specifically, if j 0 and j 1 represent the respective probabilities that an output of 0 or 1 is misclassified, then a robust version of our algorithm can handle any noise rate for which j 0 j 1 6= 1; the sample size and computation time required increase only by an ....

....majority formulas. Although omitted, a similar (though slightly more involved) algorithm can be derived for nand formulas. Our algorithm is able to handle a kind of random misclassification noise that is similar, but slightly more general than that considered by Angluin and Laird [2] and Sloan [23]. Specifically, the output of the target formula is flipped with some fixed probability that may depend on the formula s output. Thus, if the true, computed output of the formula is 0, then the learner sees 0 with probability 1 Gamma j 0 , and 1 with probability j 0 , for some quantity j 0 . ....

Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988 Workshop on Computational Learning Theory, pages 91--96, August 1988.

....framework of computational learning theory. First, it falls into the paradigm of learning with persistent noise. Here one assumes that the function f is derived from some function in the class C by adding noise to it. Typical works in this direction either tolerate only small amounts of noise [2, 37, 18, 36] (i.e. that the function is modified only at a small fraction of all possible inputs) or assume that the noise is random [1, 23, 17, 22, 30, 12, 33] i.e. that the decision of whether or not to modify the function at any given input is made by a random process) In contrast, we take the setting ....

R. H. Sloan. Types of noise in data for concept learning. COLT, 1988.

....of formulas; these are fixed sequences of instances for which every unique formula in the class induces a different labeling. We also prove that our algorithms are robust against a large amount of random misclassification noise, similar to, but slightly more general than that considered by Sloan [19] and Angluin and Laird [2] Specifically, if j 0 and j 1 represent the respective probabilities that an output of 0 or 1 is misclassified, then a robust version of our algorithm can handle any noise rate for which j 0 j 1 6= 1; the sample size and computation time required increase only by an ....

....majority formulas. Although omitted, a similar (though slightly more involved) algorithm can be derived for nand formulas. Our algorithm is able to handle a kind of random misclassification noise which is similar, but slightly more general than that considered by Angluin and Laird [2] and Sloan [19]. Specifically, the output of the target formula is flipped with some fixed probability which may depend on the formula s output. Thus, if the true, computed output of the formula is 0, then the learner sees 0 with probability 1 Gamma j 0 , and 1 with probability j 0 , for some quantity j 0 . ....

Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988 Workshop on Computational Learning Theory, pages 91--96, August 1988.

....that are very near 0 or 1; for example, days on which the sky is cloudless, or students with straight A s. This structured behavior strongly distinguishes these learning scenarios from a noisy setting, such as the one considered by Angluin and Laird [3] Kearns and Li [16] and Sloan [26]. In a model of learning with noise, the noise is typically white (that is, all inputs have either an equal probability of corruption or a probability determined by an adversary) and the noise is regarded as something an algorithm wishes to filter out in an attempt to uncover some underlying ....

Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988 Workshop on Computational Learning Theory, pages 91--96, August 1988.

....that it is consistent with some other con 1 All the results given for learning disjunctions without queries hold for k CNF and k DNF formulas as well, by standard transformations. cept in the class. It could also be thought of as a weaker version of Sloan s malicious misclassification model [Slo88] and actually is a special case of Kearns and Schapire s p concepts [KS90] though the notions of success are somewhat different. In fact, we show how Kearns and Schapire s result on learning probabilistic decision lists of decreasing probabilities can be applied to learn mixtures of k ....

.... Gamma and by c 2 with probability . This is similar to Angluin and Laird s noise model [AL88] in which there is a single target concept, but each example has fixed probability of being classified by its complement. It is also a special case of Sloan s malicious misclassification (MMC) model [Slo88] in which with probability , an adversary may decide the example s classification. Angluin and Laird describe an algorithm to learn the class of monotone disjunctions in their noise model that proceeds essentially as follows. 3 Take a large sample of data, and for each variable x i that is seen ....

[Article contains additional citation context not shown here]

Robert H. Sloan. Types of noise in data for concept learning. In David Haussler and Leonard Pitt, editors, First Workshop on Computational Learning Theory, pages 91-- 96. Morgan Kaufmann, August 1988.

....by independently inverting each attribute bit with a probability given by the noise rate. We assume throughout that the classification of each instance is always correctly reported. In the past, conflicting results on handling random attribute noise have been obtained. On the one hand, Sloan [11] has show that the best agreement rule can only tolerate very small amounts of random attribute noise suggesting that random attribute noise may be difficult to overcome. However, by using a different strategy, Shackelford and Volper [10] have obtained an algorithm that tolerates a large ....

....noise in discrete instance spaces. So these results specify the amount of noise that can be tolerated ignoring the issue of computation time. Of course, if a hypothesis minimizing disagreements can be found in polynomial time then the above techniques produce efficient learning algorithms. Sloan [11] has extended those results to the case of malicious labeling noise. Finally, as Blumer et al. mention [2] their VC dimension methods can be used to prove that this minimal disagreement method also works for handling small amounts of malicious noise in continuous instance spaces. In the case of ....

[Article contains additional citation context not shown here]

Robert H. Sloan. Types of noise in data for concept learning. In First Workshop on Computational Learning Theory, pages 91--96. Morgan Kaufmann, 1988.

No context found.

Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988.

No context found.

Robert H. Sloan. Types of noise in data for concept learning. In Proceedings of the 1988.

No context found.

Robert Sloan. Types of Noise in Data for Concept Learning. In Proceedings of the 1988 Workshop on Computational Learning Theory, pp. 91--96, MIT, ACM Press, 1988.

No context found.

Robert H. Sloan. Types of noise in data for concept learning. In First Workshop on Computational Learning Theory, pages 91--96. Morgan Kaufmann, 1988.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC