@MISC{_26.1review, author = {}, title = {26.1 Review of Last Class}, year = {} }

Share

OpenURL

Abstract

We defined PAC-learnability as follows: Definition 26.1.1 A concept class C is PAC-learnable if there is an algorithm A such that for all ǫ> 0 and δ> 0, for all distributions D on the domain, and for all target functions f ∈ C, given a sample S drawn from D, A produces a hypothesis h in some concept class C ′ such that with probability at least 1 − δ, Prx∼D [h(x) � = f(x)] < ǫ. Both the running time of algorithm A and the size of S should be poly(1/ε,1/δ,n,log |C|). Finally, h(x) should be easy to compute. Recall that we do not require h to be drawn from the same concept class as C. The motivating example was that we were trying to find functions in the concept class of 3-term DNF formulas, but doing so is NP-hard. However, by instead finding a consistent 3-DNF formula, we could still get accurate results, and the problem becomes easy. Finally, the requirement that the running time of A and the size of S should be polynomial in log |C | makes sense if one considers the fact that, since there are |C | possible functions, simply recording which we chose requires at least log |C | bits. Thus outputting the function we chose gives a lower bound on the running time of log |C|.