M. Kearns, M. Li. Learning in the presence of malicious errors. Proceedings of the 20th A.C.M. Symposium on the Theory of Computing, 1988, pp. 267-280.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....m would solve the problem of learning DNF. Observe that the class considered by Frazier et al. is a generalization of the class of DNF formulas in which all variables only appear negated. While there has been some work addressing the general issue of mislabelled training examples in the PAC model [1, 18, 28, 17], there has been little research on learning geometric concepts with noise. Auer [3] investigates exact learning of boxes where some of the counterexamples, given in response to equivalence queries, are noisy. Auer shows that box n is learnable using hypotheses from box n if and only if the ....

Michael Kearns and Ming Li. Learning in the presence of malicious errors. SIAM Journal on Computing, 22(4):807--837, August 1993.

....how well it works in practice. In general, the additional work described in section 6.6 on monitoring is a good direction for future work. The most interesting direction for work on CARD is automatic derivation of dependencies. The idea here is to use either machine learning [AL88, Kea93, BHL91, KL88, Lit89] or association rule mining [AIS93, AS94] techniques to automatically determine dependencies. This approach requires having some monitored values that indicate if a system is up or down. Then, if we can show that any time component 1 is down, component 2 is also down, but not the reverse, ....

Michael Kearns and Ming Li. Learning in the presence of malicious errors. In Proceedings of the Twentieth Annual ACM Symposium on Theory of Computing, pages 267--280, Chicago, Illinois, May 1988.

....a (somewhat large) constant factor. PIL ROBUST has another nice property; it is resistant to noise. This is very desirable if the source of sample solutions is a human oracle. In fact, the analysis of PIL ROBUST is adapted, in large part, from work in inductive learning from data with noise [ Kearns and Li, 1988 ] The model of noise which we have adopted is the malicious error model , which makes no assumptions about the nature of the errors: in particular, errors are not assumed to have a random nature, but rather to be generated by an arbitrary, and perhaps even adversary, process. 15 To be ....

Micheal Kearns and Ming Li. Learning in the presence of malicious errors. In 20th Annual Symposium on the Theory of Computing. ACM Press, 1988.

....among the computationally easiest to manipulate, which allows agnostic learning as well as a handles record noise levels (another crucial issue in Boosting [BK99] symmetric functions. Symmetric function are Boolean functions whose outputs are invariant under permutation of the input bits [KL88]. Their discriminative power is the same as linear separators: they have the same VC dimension. Computationally speaking, most the learning algorithms require record times or space when compared to many other classes of concept representations [Bou92] Ecient learning algorithms are known to learn ....

....they have the same VC dimension. Computationally speaking, most the learning algorithms require record times or space when compared to many other classes of concept representations [Bou92] Ecient learning algorithms are known to learn in the PAC or agnostic model, even under malicious noise [Bou92,KL88]. These algorithms have two common points: rst, they are purely theoretical, and studied with absolutely no experiment in the aforementioned papers or books. Second, they follow a similar, simple induction scheme which, informally, proceeds by giving to each of the n 1 possible summations the ....

[Article contains additional citation context not shown here]

M.J. Kearns and M. Li. Learning in the presence of malicious errors. In ACM Symposium on the Theory of Computing, pages 267-280, 1988.

....examples given to the learning algorithm were noisy. Two popular noise models for both theoretical and experimental research are the classification noise model introduced by Angluin and Laird [2, 21] and the malicious error model introduced by Valiant [35] and further studied by Kearns and Li [20]. In the classification noise model, each example received by the learner is mislabelled randomly and independently with some fixed probability. In the malicious error model, an adversary is allowed, with some fixed probability, to substitute a labelled example of his choosing for the labelled ....

....demonstrated that calls to a statistics oracle can be simulated, with high probability, by a procedure which draws a suificiently large sample from a malicious error oracle. Thus, nearly every PAC learning algorithm can be transformed into one which tolerates malicious errors. While Kearns and Li [20] had previously demonstrated a general technique for converting a PAC learning algorithm into one which tolerates small amounts of malicious error, the results obtained by appealing to SQ are better in some interesting cases [9] While greatly expanding the function classes known to be learnable ....

[Article contains additional citation context not shown here]

Michael Kearns and Ming Li. Learning in the presence of malicious errors. In Proceedings of the 20 tn Annual A CM Symposium on Theory of Computing, Chicago, Illinois, May 1988.

....examples given to the learning algorithm were noisy. Two popular noise models for both theoretical and experimental research are the classification noise model introduced by Angluin and Laird [2, 21] and the malicious error model introduced by Valiant [35] and further studied by Kearns and Li [20]. In the classification noise model, each example received by the learner is mislabelled randomly and independently with some fixed probability. In the malicious error model, an adversary is allowed, with some fixed probability, to substitute a labelled example of his choosing for the labelled ....

....demonstrated that calls to a statistics oracle can be simulated, with high probability, by a procedure which draws a sufficiently large sample from a malicious error oracle. Thus, nearly every PAC learning algorithm can be transformed into one which tolerates malicious errors. While Kearns and Li [20] had previously demonstrated a general technique for converting a PAC learning algorithm into one which tolerates small amounts of malicious error, the results obtained by appealing to SQ are better in some interesting cases [9] While greatly expanding the function classes known to be learnable ....

[Article contains additional citation context not shown here]

Michael Kearns and Ming Li. Learning in the presence of malicious errors. In Proceedings of the Twentieth Annual ACM Symposium on Theory of Computing, pages 267--280, Chicago, Illinois, May 1988.

....We model an attack as the addition of noise to the training data. In general, this noise could be associated with either the attributes, the class or both. We focus exclusively on class noise, and defer attribute noise to future work. We are not concerned with malicious noise as defined by [6], because we can safely assume that the attacking agent is not omniscient (eg, the agents can not directly inspect any ratings except their own) We model attacks with a relatively benign noise model that we call biased class noise. This model is characterized by terms parameters: the noise rate ....

M. Kearns and M. Li. Learning in the presence of malicious errors. In Proc. ACM Symp. Theory of Computing, 1988.

....noise free attributes only if the attacker knows the probability distribution over the space of ratings for all products. We are modelling an attacker that therefore lies in the middle of the spectrum between ignorance and omniscience. We are not concerned with malicious class noise as defined by [13]. Rather, we model attacks with a relatively benign noise model that we 6 call biased class noise. This model is characterised by two parameters: the noise rate #, and the class bias . Noise is added according to the following process. First, an instance is generated according to the underlying ....

M. Kearns and M. Li. Learning in the presence of malicious errors. In Proc. ACM Symp. Theory of Computing, 1988.

....DNF formulas [13, 20] etc. For all of these classes it is known (using information theoretic arguments) that neither membership queries nor equivalence queries alone suffice. Although different from the goals of our work, there has been work on learning when the examples may be mislabeled [3, 32, 41, 29] and when there is attribute noise [40, 23, 33] There has also been some work in which the answers to membership queries are noisy or missing [37, 7, 42, 6, 12] Although we can get a response from our membership oracle, this response is not adversarially generated to model an inconclusive ....

M. Kearns and M. Li. Learning in the presence of malicious errors. SIAM J. Comput., 22:807--837, 1993.

....has to make the same decision in each scenario. However, since any decision is bad for at least one scenario in Q, there must be a scenario in which the learner fails. The exact formulation of this scheme depends on the given learning model. Such a scheme has been rst used by Kearns and Li [20] (and later by others, cf. 16] in proving the information theoretic upper bound on the rate of tolerable malicious noise. Speci cally, they show that by maliciously corrupting an 1 fraction of the learner s sample, there are two di erent scenarios which induce the same distribution over ....

Kearns, M. J. and Li, M. (1993). Learning in the presence of malicious errors. SIAM J. Comput., 22:807-837.

.... In this malicious PAC model, each training example given to the learner is independently replaced, with fixed probability j, by an adversarially chosen one (which may or may not be consistent with the f0; 1gvalued target function) In their comprehensive investigation of malicious PAC learning [4], Kearns and Li show that a malicious noise rate j = 1 ) can make statistically indistinguishable two target functions that differ on a subset of the domain whose probability measure is at least . This implies that with this noise rate no learner can generate hypotheses that are good in ....

....3.4) by showing that at least order of = Delta 2 d= Delta examples are needed to PAC learn, with accuracy and tolerating a malicious noise rate j = 1 ) Gamma Delta, every class of f0; 1g valued functions of VC dimension d. Our proof combines, in an original way, techniques from [2, 4, 10] and uses some new estimates of the tails of the binomial distribution that may be of independent interest. We then prove that this lower bound cannot be improved in general. Namely, we show (Theorems 3.18 and 3.10) that there is an algorithm RMD (for Randomized Minimum Disagreement) that, for ....

[Article contains additional citation context not shown here]

M. Kearns and M. Li. Learning in the presence of malicious errors. SIAM J. Comput., 22:807--837, 1993.

....framework of computational learning theory. First, it falls within the framework of learning with persistent noise. Here one assumes that the function f is derived from some function in the class C by adding noise to it. Typical works in this direction either tolerate only small amounts of noise [2, 42, 21, 39] (i.e. that the function is modified only at a small fraction of all possible inputs) or assume that the noise is random [1, 26, 20, 25, 33, 13, 36] i.e. that the decision of whether or not to modify the function at any given input is made by a random process) In contrast, we take the setting ....

Michael Kearns and Ming Li. Learning in the presence of malicious errors. SIAM Journal on Computing, 22(4): 807--837, August 1993.

....the presense of malicious errors, while retaining the Omega Gamma ) error tolerance present in the known simulation. Note that a linear dependence on 1= is optimal for both sample complexities [5, 9] and a linear dependence on is optimal for the error tolerance in the malicious error model [15]. 2 Background Before presenting the new model, we give formal definitions of the other learning models used throughout this paper. We begin by defining the example based PAC learning model as well as the classification noise and malicious error variants. We then define the standard additive ....

.... error statistical queries using a sample size whose dependence on is roughly optimal, whereas the use of additive error statistical queries yielded an additional factor of 1= In Corollary 3, these bounds are within logarithmic factors of both the O( maximum allowable malicious error rate [15] and the Omega Gammae = lower bound on the sample complexity for noise free PAC learning [9] We also note that in this malicious error tolerant PAC simulation, the sample, time, space and hypothesis size complexities are asymptotically identical to the corresponding complexities in our ....

Michael Kearns and Ming Li. Learning in the presence of malicious errors. SIAM Journal on Computing, 22(4):807--837, 1993.

....under this selected projection. From a philosophical point of view, one may attribute all randomness (or noise phenomena) in nature to the effect of some invisible deterministic variables on the observable environment. As such, models concerning noisy information like that of Kearns and Li [18], as well as those dealing with probabilistic concepts like the Kearns Schapire p concepts [19] may be viewed as special cases of our scenario. Another related model is the model of switching concepts introduced by Blum and Chalasani [10] In that model a probabilistic behavior of the unknown ....

....classification of the examples supplied to the learner, by a random i.i.d. variable that has some fixed probability j of assuming the inverse of the true classification value, and with probability (1 Gamma j) assumes the correct classification value [21, 28, 2] for a stronger model of noise see [18, 12]) The task of providing methods by which efficient learning algorithms can be strengthened to noise tolerant ones is one of the most important in computational learning theory. As this task seems to be very difficult, one naturally seeks partial solutions. A natural approach is to consider ....

Michael J. Kearns and Ming Li. Learning in the presence of malicious errors. In SIAM Journal of Computing, volume 22, pages 807--837, 1993.

....results for agnostically learning simple classes address the problem of finding a hypothesis that maximizes the number of agreements (rather than just approximately maximizing it) Angluin and Laird [2] showed that maximizing agreements with monotone mono5 mials is NP hard. Kearns and Li [18] established a similar result for general monomials, and Hoffgen, Simon and Van Horn [16] for half spaces. One may argue, however, that for all practical purposes, a learner may be considered successful if it can produce a hypothesis with accuracy within a small constant factor of the best ....

....led to the next level of hardness of learning results, showing that guarantees of this type cannot be achieved. Judd [17] shows NP hardness results for an approximate sample error minimization problem for certain linear threshold networks with many outputs. Combining a reduction of Kearns and Li [18] with recent results on the hardness of approximating set cover [10] implies that, unless NP DTIME(n log log n ) finding a monomial that has a ratio of misclassifications within a factor o(log n) of the minimum is not possible. Arora, Babai, Stern and Sweedyk [3] showed that is NP hard to ....

[Article contains additional citation context not shown here]

M. Kearns and M. Li. Learning in the presence of malicious errors. SIAM Journal on Computing, 22(4):807--837, 1993.

....who can modify both example points and labels in an arbitrary fashion (a detailed description of the model is given in Section 3) The frequency of such corrupted examples is known as the malicious noise rate. Learning in the presence of malicious noise is in general quite dicult. Kearns and Li [16] have shown that for many concept classes it is impossible to learn to accuracy if the malicious noise rate exceeds 1 : In fact, for many interesting concept classes (such as the class of linear threshold functions) the best ecient algorithms known can only tolerate malicious noise rates ....

.... for the target concept sign(u x) the vector u is either (1; 0) or (0; 1) The four points in f1; 1g 2 are classi ed in all four possible ways by these two concepts, and hence the concept class consisting of these two concepts is a distinct concept class as de ned by Kearns and Li [16]. It is clear that every example belongs to B1 (1) i.e. R = 1) that kuk 1 = 1; and that any distribution D over f1; 1g 2 is 1 good for u (i.e. 1) Consider the following algorithm for this restricted learning problem: Draw from EX MAL (u; D) a sample S = hx 1 ; y 1 i; hx ....

[Article contains additional citation context not shown here]

M. Kearns and M. Li. Learning in the presence of malicious errors, SIAM J. Comput. 22(4) (1993), 807-837.

.... whenever the coin shows H , which happens with probability j, the classification of the example is flipped and so the algorithm is provided with the, wrongly classified, example (x; 1 Gamma c t (x) Another (stronger) model, called the Malicious Noise model, was introduced in [23] revisited in [17], and was further studied in [8, 10, 11, 20] In this model the adversary, whenever the j biased coin shows H , can replace the example (x; c t (x) by some arbitrary pair (x 0 ; b) where x 0 is any point in the input space and b is a boolean value. Note that this in particular gives the ....

M. J. Kearns and M. Li, "Learning in the Presence of Malicious Errors", SIAM J. on Computing, 22:807837, 1993.

....framework of computational learning theory. First, it falls within the framework of learning with persistent noise. Here one assumes that the function f is derived from some function in the class C by adding noise to it. Typical works in this direction either tolerate only small amounts of noise [2, 41, 21, 39] (i.e. that the function is modi ed only at a small fraction of all possible inputs) or assume that the noise is random [1, 26, 20, 25, 33, 13, 36] i.e. that the decision of whether or not to modify the function at any given input is made by a random process) In contrast, we take the setting ....

Michael Kearns and Ming Li. Learning in the presence of malicious errors. SIAM Journal on Computing, 22(4): 807-837, August 1993. 37

No context found.

M. Kearns, M. Li. Learning in the presence of malicious errors. Proceedings of the 20th A.C.M. Symposium on the Theory of Computing, 1988, pp. 267-280.

No context found.

Michael Kearns and Ming Li. Learning in the presence of malicious errors. In Proceedings of the Twentieth Annual ACM Symposium on Theory of Computing, pages 267--280, May 1988. To appear, SIAM Journal on Computing.

No context found.

Michael Kearns and Ming Li. Learning in the presence of malicious errors. In Proceedings of the Twentieth Annual ACM Symposium on Theory of Computing, pages 267--280, May 1988. To appear, SIAM Journal on Computing.

No context found.

Kearns, M., M. Li, "Learning in the presence of malicious errors", 20th ACM Symposium on the Theory of Computing, Chicago, IL, 1988, pp. 267-280.

No context found.

M.J. Kearns, M. Li. Learning in the Presence of Malicious Errors. SIAM Journal on Computing, 22(4):807-837, 1993. 67

No context found.

Michael Kearns and Ming Li. Learning in the presence of malicious errors. SIAM Journal on Computing, 22(4):807-- 837, August 1993.

No context found.

KEARNS, M., AND LI, M. (1988), Learning in the Presence of Malicious Errors. in "Proceedings of the 20th Symposium on Theory of Computing", pp 267--280.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC