Roni Khardon. On using the Fourier transform to learn disjoint DNF. Unpublished manuscript, September 1993.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....this polynomial has the claimed properties. Noting that g = 2P 0 1 completes the proof. Theorem 9) By restricting the size of terms in the SAT k DNF s considered we can extend the above to a distribution free learning result (this generalizes a similar result for SAT 1 (disjoint) DNF by Khardon [11]) Theorem 10 For any k, the class of SAT k O(log s) DNF formulas of s terms can be learned exactly by a deterministic learning algorithm which uses membership queries and runs in time polynomial in n, s k , 1= k , and log(1= 5 Characterizing Learnability in the Statistical Query Model ....

Roni Khardon. On using the Fourier transform to learn disjoint DNF. Unpublished Manuscript, 9 1993.

....in his seminal 1984 paper introducing the PAC learning model [37] more than fteen years later this question is widely regarded as one of the most important open problems in learning theory. While many partial results have been given for restricted versions of the DNF learning problem (see e.g. [8, 9, 21, 23, 24, 26, 27, 33, 38, 39]) the diculty of the unrestricted DNF learning problem is evidenced by the fact that, prior to the current work, only two algorithms were known which improve on the naive 2 n time bound [11, 36] The rst subexponential time algorithm for learning DNF was due to Bshouty [11] who gave an ....

R. Khardon. On using the Fourier transform to learn disjoint DNF. Inf. Proc. Lett. 49 (1994), 219-222.

....such as the uniform distribution, rather than all distributions; requiring that the learner succeed only for restricted subclasses of DNF formulae such as monotone DNF with a bounded number of terms. A SAT k DNF is a DNF in which each truth assignment satis es at most k terms. Khardon [22] gave a polynomial time membership query algorithm for learning polynomial size SAT 1 DNF under the uniform distribution; this result was later strengthened by Blum et al. 4] to SAT k DNF for any constant k: Bellare [6] gave a polynomial time membership query algorithm for learning O(log ....

....s m = x 1 xm : Then m 2B 2 2 ln 2 implies that Pr h s m m p i : 2. 1 The Learning Model Our learning model is a distribution speci c version of Valiant s Probably Approximately Correct (PAC) model [30] which has been studied by many researchers, e.g. [4, 6, 11 13, 16, 18, 22, 24, 26, 27, 31, 32]. Let C be a class of Boolean functions over f0; 1g n ; let D be a probability distribution over f0; 1g n ; and let f 2 C be an unknown target function. A learning algorithm A for C takes as input an accuracy parameter 0 1 and a con dence parameter 0 1: During its execution the ....

R. Khardon. On using the Fourier transform to learn disjoint DNF, Information Processing Letters 49 (1994), 219-222.

.... that a boolean decision tree of depth k can be represented in MUL (n; k; 2 k ) each leaf in the decision tree defines a term and the function is the sum of all terms) and that a j disjoint k DNF of size t can be represented in MUL (n; k(j Gamma 1) t j Gamma1 ) See for example [K94]. So for constant k and d = O(log n) the number of terms is polynomial. For a DNF and multivariate polynomial, f , we define size(f) to be the number of terms in f . For a decision tree the size will be the number of leaves in the tree. A product distribution is a distribution D that satisfies ....

R. Khardon. On using the Fourier transform to learn disjoint DNF. Information Processing Letters, 49 (05), pp. 219-222 (1994).

....powerful membership oracle. 23 Rivest [Riv87] showed that the class of decision lists, which contains both k DNF and k CNF, is also PAC learnable. Given the added advantage of membership queries, Blum and Rudich [BR92] proved the learnability of the class of (log n) term DNF. Recently, Khardon [Kha94] used Fourier techniques to give an alternate proof of this result, for learning over the uniform distribution. Another important restriction of DNF is the class of monotone DNF. Kearns et al. KLPV87] have shown that under the PAC model, learning DNF reduces to learning monotone DNF. That is, ....

....and Bshouty [Bsh93] has given an efficient algorithm for learning decision trees over any distribution, with membership queries. He has further shown that any Boolean function is learnable (with membership queries) in time polynomial in the larger of its DNF and CNF sizes. Finally, Khardon [Kha94] uses Fourier analysis to give a membership query algorithm for efficiently learning disjoint DNF, which strictly contains the class of decision trees. We will present this result later in the chapter. Much of the evidence regarding the learnability of the general class of DNF expressions is ....

[Article contains additional citation context not shown here]

Roni Khardon. On using the Fourier transform to learn disjoint DNF. Information Processing Letters, 49(1994):219--222, 1994.

....on examples, such as the uniform distribution, rather than all distributions; ffl requiring that the learner succeed only for restricted subclasses of DNF formulae such as DNF with a bounded number of terms. A SAT k DNF is a DNF in which each truth assignment satisfies at most k terms. Khardon [19] gave a polynomial time membership query algorithm for learning polynomial size SAT 1 DNF under the uniform distribution; this result was later strengthened by Blum et al. 3] to SAT k DNF for any constant k: Bellare [5] gave a polynomial time membership query algorithm for learning O(log n) ....

....: Then m 2B 2 ffl 2 ln 2 ffi implies that Pr fi fi fi fi s m m Gamma p fi fi fi fi ffl ffi: 2. 1 The Learning Model Our learning model is a distribution specific version of Valiant s Probably Approximately Correct (PAC) model [26] and has been studied by many researchers, e.g. [3, 5, 8, 9, 10, 13, 15, 19, 21, 22, 23, 27, 28]. Let C be a class of Boolean functions over f0; 1g n ; let D be a probability distribution over f0; 1g n ; and let f 2 C be an unknown target function. A learning algorithm A for C takes as input an accuracy parameter 0 ffl 1 and a confidence parameter 0 ffi 1: During its execution ....

R. Khardon. On using the Fourier transform to learn disjoint DNF, Inf. Proc. Lett. 49 (1994), 219-222.

.... Since then, many theoretical studies have investigated the learnability of DNF or subclasses [ Kearns et al. 1987; Pillaipakkamnatt and Raghavan, 1994; Aizenstein and Pitt, 1995; Aizenstein and Pitt, 1992; Berggren, 1993; Blum et al. 1994; Bshouty et al. 1995; Goldman and Mathias, 1992; Khardon, 1994; Mansour, 1992; Pitt and Valiant, 1988 ] Simultaneously, many programs for machine learning were designed to learn efficiently from examples. In each of these programs, the algorithms has access to a learning sample and tries to build a small function approximating as best as possible the ....

R. Khardon. On using the fourier transform to learn disjoint DNF. Information Processing Letters, pages 219--222, 1994.

....in his seminal 1984 paper introducing the PAC learning model [36] more than fifteen years later this question is widely regarded as one of the most important open problems in learning theory. While many partial results have been given for restricted versions of the DNF learning problem (see e.g. [8, 9, 21, 23, 24, 26, 27, 32, 37, 38]) the difficulty of the unrestricted DNF learning problem is evidenced by the fact that, prior to the current work, only two algorithms were known which improve on the naive 2 n time bound [11, 35] The first subexponential time algorithm for learning DNF was due to Bshouty [11] who gave an ....

R. Khardon. On using the Fourier transform to learn disjoint DNF. Inf. Proc. Lett. 49 (1994), 219-222.

....set of instances x chosen at random from D, each one labeled with whether it is a positive or negative instance. We chose the PAC learning model for demonstrating that ARVPs are polynomially learnable because the PAC model is a distribution free approach (in contrast to more restricted approaches [6, 7, 8, 9] that make specific assumptions about the distributions of input examples) In the PAC model, so long as the distribution D is the same during learning and testing, the algorithm should give a good result, regardless of the distribution type. Another alternative is the exact learning model ....

R. Khardon. On using the Fourier transform to learn disjoint DNF. Information Processing Letters, 49(5):219--222, 1994.

.... in the SAT k DNF s considered and using exact reconstruction and derandomization techniques similar to those of Kushilevitz and Mansour [16] we can extend the above to a deterministic, distributionindependent learning result (this generalizes a similar result for SAT 1 (disjoint) DNF by Khardon [14]) Theorem 10 For any k, the class of SAT k O(log s) DNF formulas of s terms can be learned exactly by a deterministic learning algorithm which uses membership queries and runs in time polynomial in n and s k . 4.4 Learning c PT 1 In this section we generalize our weak learning result for ....

Roni Khardon. On using the Fourier transform to learn disjoint DNF. Unpublished manuscript, September 1993.

No context found.

R. Khardon. On using the Fourier transform to learn disjoint DNF. Information Processing Letters, 49:219-222, 1994.

....a polynomial time algorithm for learning arbitrary DNF formulas in the PAC model with membership queries, provided that the error is measured with respect to the uniform distribution. This extends results on learning decision trees, disjoint DNF and Satisfy j DNF formulas in the same model [KM93, Kha94, BFJ 94] On the negative side, some recent hardness results for learning DNF in restricted models apply for disjoint DNF as well: Blum et al. BFJ 94] prove a hardness result for learning log n disjoint DNF in the statistical queries model (note that in this model the learner does not ....

R. Khardon. On using the Fourier transform to learn disjoint DNF. Information Processing Letters, 49(5):219--222, March 1994.

....a polynomial time algorithm for learning arbitrary DNF formulas in the PAC model with membership queries, provided that the error is measured with respect to the uniform distribution. This extends results on learning decision trees, disjoint DNF and Satisfy j DNF formulas in the same model [KM93, Kha94, BFJ 94] On the negative side, some recent hardness results for learning DNF in restricted models apply for disjoint DNF as well: Blum et al. BFJ 94] prove a hardness result for learning log n disjoint DNF in the statistical queries model (note that in this model the learner does not ....

R. Khardon. On using the Fourier transform to learn disjoint DNF. Information Processing Letters, 49(5):219--222, March 1994.

No context found.

Roni Khardon. On using the Fourier transform to learn disjoint DNF. Unpublished manuscript, September 1993.

No context found.

R. Khardon. On using the fourier transform to learn disjoint DNF. Information Processing Letters, 49:219-222, 1994.

No context found.

R. Khardon. On using the Fourier transform to learn disjoint DNF. Information Processing Letters, 49:219--222, 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC