Results 1 
4 of
4
Hardness of learning halfspaces with noise
 In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science
, 2006
"... Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noisefree case, when a halfspace consistent with all the training examples exists, the problem can be solved in polynomial time using linear programming. However ..."
Abstract

Cited by 40 (3 self)
 Add to MetaCart
(Show Context)
Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noisefree case, when a halfspace consistent with all the training examples exists, the problem can be solved in polynomial time using linear programming. However, under the promise that a halfspace consistent with a fraction (1 − ε) of the examples exists (for some small constant ε> 0), it was not known how to efficiently find a halfspace that is correct on even 51 % of the examples. Nor was a hardness result that ruled out getting agreement on more than 99.9 % of the examples known. In this work, we close this gap in our understanding, and prove that even a tiny amount of worstcase noise makes the problem of learning halfspaces intractable in a strong sense. Specifically, for arbitrary ε, δ> 0, we prove that given a set of exampleslabel pairs from the hypercube a fraction (1 − ε) of which can be explained by a halfspace, it is NPhard to find a halfspace that correctly labels a fraction (1/2 + δ) of the examples. The hardness result is tight since it is trivial to get agreement on 1/2 the examples. In learning theory parlance, we prove that weak proper agnostic learning of halfspaces is hard. This settles a question that was raised by Blum et al. in their work on learning halfspaces in the presence of random classification noise [10], and in some more recent works as well. Along the way, we also obtain a strong hardness result for another basic computational problem: solving a linear system over the rationals. 1
HARDNESS OF SOLVING SPARSE OVERDETERMINED LINEAR SYSTEMS: A 3QUERY PCP OVER INTEGERS
"... Abstract. A classic result due to H˚astad established that for every constant ε> 0, given an overdetermined system of linear equations over a finite field Fq where each equation depends on exactly 3 variables and at least a fraction (1 − ε) of the equations can be satisfied, it is NPhard to sati ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract. A classic result due to H˚astad established that for every constant ε> 0, given an overdetermined system of linear equations over a finite field Fq where each equation depends on exactly 3 variables and at least a fraction (1 − ε) of the equations can be satisfied, it is NPhard to satisfy even a fraction ` 1 q + ε ´ of the equations. In this work, we prove the analog of H˚astad’s result for equations over the integers (as well as the reals). Formally, we prove that for every ε, δ> 0, given a system of linear equations with integer coefficients where each equation is on 3 variables, it is NPhard to distinguish between the following two cases: (i) There is an assignment of integer values to the variables that satisfies at least a fraction (1 − ε) of the equations, and (ii) No assignment even of real values to the variables satisfies more than a fraction δ of the equations. 1.
ABSTRACT A 3Query PCP over Integers
"... A classic result due to H˚astad [11] established that for every constant ε> 0, given an overdetermined system of linear equations over a finite field Fq where each equation depends on exactly 3 variables and at least a fraction (1 − ε) of the equations “ can be ” satisfied, it is NPhard to satis ..."
Abstract
 Add to MetaCart
A classic result due to H˚astad [11] established that for every constant ε> 0, given an overdetermined system of linear equations over a finite field Fq where each equation depends on exactly 3 variables and at least a fraction (1 − ε) of the equations “ can be ” satisfied, it is NPhard to satisfy even a 1 fraction + ε of the equations. q In this work, we prove the analog of H˚astad’s result for equations over the integers (as well as the reals). Formally, we prove that for every ε, δ> 0, given a system of linear equations with integer coefficients where each equation is on 3 variables, it is NPhard to distinguish between the following two cases: (i) There is an assignment of integer values to the variables that satisfies at least a fraction (1−ε) of the equations, and (ii) No assignment even of real values to the variables satisfies more than a fraction δ of the equations.