E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In Proc. of 38th Symposium on Foundations of Computer Science, pages 514--523, Miami, FL, 1997. IEEE.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... in both the PAC model of learning from random examples and in the model of exact learning from equivalence queries [10, 31] Re nements of the basic linear programming approach have led to polynomial time algorithms for PAC learning linear threshold functions in the presence of classi cation noise [7, 14]. Much attention has also been given to fast, simple heuristics, most notably the Winnow and Perceptron algorithms, for learning linear threshold functions [12, 18, 25, 29, 34, 35] 1.2 Learning DNF Another intensively studied problem in computational learning theory, which has met with less ....

....value is O(n 1=3 log s) Applying Fact 7 gives our main DNF learning result: Corollary 10. The class of polynomial size DNF can be learned (in both the PAC model and the model of exact learning from equivalence queries) in time 2 O(n 1=3 log 2 n) Remark: Several algorithms are known [7, 14] for PAC learning linear threshold functions over f0; 1g n in the presence of classi cation noise in time poly(n) It follows that our time bounds for learning DNF continue to hold in the presence of classi cation noise. Corollary 11. The n 1=3 ) lower bound given by Minsky and Papert for ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time, in \Proc. 38th Symp. on Found. of Comp. Sci." (1997), 514-523.

....theory. Many algorithms have been analyzed in the noise free setting where all examples are labeled correctly (see [8, 10, 25] for some representative approaches) and powerful algorithms have recently been developed for the classification noise model where the labels of examples may be corrupted [9, 12]. While such algorithms and analyses are theoretically interesting, their utility for practical learning problems is limited by the fact that in the real world noise may corrupt example points themselves as well as example labels. A stronger, more realistic model of malicious noise was earlier ....

....gap between our lower bound of Omega Gamma ffl Delta R ) and the Kearns Li upper bound of O(ffl) on the malicious noise rate which can be tolerated by an efficient linear threshold learning algorithm. Another goal is to develop malicious noise tolerant algorithms which, like the algorithms of [9, 10, 12], can learn efficiently under arbitrary distributions. In another direction, if weak learning algorithms which tolerate malicious noise can be developed for other concept classes then Smoothboost could be applied to obtain noise tolerant strong learning algorithms for these concept classes. ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time, in "Proc. 38th Symp. on Found. of Comp. Sci." (1997), 514-523.

.... both the PAC model of learning from random examples and in the model of exact learning from equivalence queries [10, 30] Refinements of the basic linear programming approach have led to polynomialtime algorithms for PAC learning linear threshold functions in the presence of classification noise [7, 14]. Much attention has also been given to fast, simple heuristics, most notably the Winnow and Perceptron algorithms, for learning linear threshold functions [12, 18, 25, 29, 33, 34] 1.2 Learning DNF Another intensively studied problem in computational learning theory, which has met with less ....

.... O( n log n) 1=3 log s: Applying Fact 8 gives our main DNF learning result: Corollary 14 The class of polynomial size DNF can be learned (in both the PAC model and the model of exact learning from equivalence queries) in time 2 O(n 1=3 log 7=3 n) Remark: Several algorithms are known [7, 14] for PAC learning linear threshold functions over f0; 1g n in the presence of classification noise in time poly(n) It follows that our time bounds for learning DNF continue to hold in the presence of classification noise. Corollary 15 The Omega Gamma n 1=3 ) lower bound given by Minsky and ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time, in "Proc. 38th Symp. on Found. of Comp. Sci." (1997), 514-523.

.... [22] gave a simple algorithm for learning origin centered halfspaces under the uniform distribution in the presence of classification noise; his algorithm requires O( n 2 ffl 2 (1 Gamma2j) 2 ) examples and runs in time O( n 3 ffl 2 (1 Gamma2j) 2 ) Blum et al. 9] and Cohen [13] have recently given polynomial time algorithms for learning halfspaces in the presence of classification noise under an arbitrary distribution. Their algorithms have time complexity at least O( n 14 ffl 2 (1 Gamma2j) 2 ) though, and hence are not particularly efficient for the uniform ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In "38th Symposium on Foundations of Computer Science,", pages 514-523, 1997.

....xj. Using lemma 3, and theorems 2, 3, we have the main result of this section. Theorem 4 An robust half space in R n can be PAC learned using O( log 1 2 ) examples in O( n 2 ) time. The Perceptron Algorithm is known to be tolerant to various types of classification noise [4, 2, 6]. It is a straightforward consequence that these properties continue to hold for our algorithm. In the concluding section we discuss straightforward bounds for agnostic learning. 4.2 Intersections of half spaces The next problem we consider is learning an intersection of m half spaces in R n , ....

E. Cohen, "Learning noisy perceptrons by a perceptron in polynomial time," Proc. of the 38th IEEE Foundations of Computer Science, 1997.

....neural networks. For concept learning in which some linear threshold function is a perfect classifier, mistake bounds are known for the Perceptron algorithm [22, 25] and the Winnow and Weighted Majority algorithms [18, 19, 21] There are also results for these algorithms for various types of noise [3, 4, 5, 6, 10, 20]. However, these previous results do not characterize the behavior of these algorithms over any sequence of examples. This paper shows that minimizing the absolute loss characterizes the online behavior of two algorithms for learning linear threshold functions: the Perceptron algorithm and the ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In Proc. 38th IEEE Annual Symposium on Foundations of Computer Science, 1997.

....construct more powerful boosting based PAC algorithms for linear threshold functions. All of the algorithms discussed in this paper have an inverse quadratic dependence on the separation parameter ffi u;X ; linear programming based algorithms for learning linear threshold functions (see, e.g. [6, 7, 9, 29, 30]) do not have such a dependence. Is there a natural boosting based PAC algorithm for linear threshold functions with performance bounds similar to those of the linear programming based algorithms 7 ACKNOWLEDGEMENTS We warmly thank Les Valiant for helpful comments and suggestions. ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time, in "Proc. 38th Symp. on Found. of Comp. Sci." (1997), 514-523.

....algorithms. For instance, linearly separable dichotomies can be learned by threshold unit perceptrons in polynomial time, from algorithms based on linear programming and involving a large number of parameters. 2 However, the Perceptron learning algorithm [22] has not a polynomial behavior [7] in spite of its convergence theorem. The complexity of more elaborate learning rules, such as backpropagation, is not even addressable since their convergence is not ensured. A polynomial time algorithm has been proposed for constructing and learning a very special class of multilayer perceptrons ....

....special class of multilayer perceptrons [23] the learning rule being a clustering algorithm. The multilayer network architecture which can be derived is rather artificial and the network size (number of units and weights) is very large w.r.t. to the sample size. Several recent papers [3] 24] [7] propose polynomial time algorithms for learning in perceptrons or multilayer networks, but only for very specific problems (e.g. noisy linearly separable dichotomies; intersections of half spaces) But, as far as we know, no polynomial time algorithm has been proposed for learning general data ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In 38th Symposium on the Foundations of Computer Science, pages 514--523, 1997.

.... there are some powerful general techniques for making learning algorithms cope with noise in some generality [16] For the problem of learning linear separators there exist theoretical results that show that there is no fundamental computational impediment to overcoming random classification noise [5, 8]. Currently somewhat complex algorithms are needed to establish this rigorously. In practice, fortunately, natural algorithms such as the perceptron algorithm and Winnow, or the linear discriminant algorithm, behave well on natural data sets which are often noisy, and for which there is no a ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In Proc 38th IEEE Symp. on Foundation of Computer Science, pages 514--523, 1997.

....each example is mislabeled (i.e. differs from the target LTF) with the same probability [10] This paper shows that LTFs are polynomially learnable in the presence of classification noise if there is a sufficient separation between the examples and the target hyperplane. Blum et al. 2] and Cohen [6] improve on this result by demonstrating that a large separation is not necessary for polynomial learning in this case. All of the above three special cases have the following characteristic in common. If one example is more likely to be mislabeled than another example (i.e. less likely to agree ....

....a sufficient separation between the examples and the target hyperplane. As might be expected for a more general case, considerably more examples (but still a polynomial number) appear to be needed as compared to classification noise. It is unclear whether the results of Blum et al. 2] and Cohen [6] can be modified for monotonic noise. 1 Bylander [5] used the term probabilistically consistent to describe this type of noise. This paper adopts the hopefully more accurate term monotonic noise. 2 Definitions An example x is a vector of n real numerical attributes, x 2 n and kxk 2 ....

[Article contains additional citation context not shown here]

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In Proc. 38th IEEE Annual Symposium on Foundations of Computer Science, 1997.

....labels all points in the new sample, and so a standard linear programming algorithm can now be applied to find a consistent linear threshold function for that data. VC dimension arguments then imply that this new hypothesis likely has low true error. An alternative approach is described by Cohen [Coh97]. A big open question is whether weak learning is possible in the presence of adversarial noise. For instance, given a set of examples that are nearly (90 ) linearly separable, can one find a linear threshold function that correctly classifies at least a 1=2 1=poly(n; b) fraction More ....

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In Proceedings of the 38th Annual Symposium on Foundations of Computer Science, pages 514-523, October 1997.

No context found.

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In Proc. of 38th Symposium on Foundations of Computer Science, pages 514--523, Miami, FL, 1997. IEEE.

No context found.

E. Cohen. Learning Noisy Perceptron by a Perceptron in Polynomial Time. In Proceedings of the 38th Annual Symposium on Foundations of Computer Science, pages 514-523, 1997.

No context found.

E. Cohen, Learning noisy perceptrons by a perceptron in polynomial time, Proceedings of the Annual IEEE Symposium on the Foundations of Computer Science, 514--523, 1997.

No context found.

E. Cohen. Learning Noisy Perceptron by a Perceptron in Polynomial Time. In Proceedings of the 38th Annual Symposium on Foundations of Computer Science, pages 514-523, 1997.

No context found.

Edith Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In 38th Annual Symposium on Foundations of Computer Science, 1997.

No context found.

E. Cohen. Learning Noisy Perceptron by a Perceptron in Polynomial Time. In Proceedings of the 38th Annual Symposium on Foundations of Computer Science, pages 514-523, 1997.

No context found.

E. Cohen. Learning Noisy Perceptron by a Perceptron in Polynomial Time. In Proceedings of the 38th Annual Symposium on Foundations of Computer Science, pages 514-523, 1997.

No context found.

E. Cohen. Learning noisy perceptrons by a perceptron in polynomial time. In 38th Symposium on the Foundations of Computer Science, pages 514-- 523, 1997.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC