J. C. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. Journal of Computer and System Sciences, 55:414--440, 1997.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....contain values from some field. 1 Introduction Two techniques were used in the literature for PAC learning decision tree with membership queries. Kushilevitz and Mansour gave in [KM93] a technique for learning decision trees under the uniform distribution via the Fourier Spectrum. Jackson in [J94] extended the result to learning DNF under the uniform distribution. The output hypothesis is a majority of parities. Bshouty gave in [Bs94] a technique for learning decision trees under any distribution via the Monotone Theory. Schapire and Sellie gave in [SS93] a Lattice based algorithm for ....

....(with monotone terms) are PAC learnable with membership queries under any distribution [SS93] Our contribution is to show the same when the terms are not nesasary monotone. It is also known that any DNF is PAC learnable with membership queries under constant bounded product distribution [J94]. In [J94] the output hypothesis is a majority of parities. Our contribution for j disjoint DNF is to use an output hypothesis that is a parity of terms and to show that the output hypothesis is an ffl approximation of the target against any constant bounded distribution. Also, the learnability of ....

[Article contains additional citation context not shown here]

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proceeding of the 35th Annual Symposium on Foundations of Computer Science, 1994.

....decision trees) In particular, we present a family of functions fF n g that have short (poly(n) read 2 disjoint DNF formulas but require CNF formulas of size 2 Omega Gamma . Other results concerning the learnability of disjoint DNF have also been obtained. On the positive side, Jackson [Jac94] gives a polynomial time algorithm for learning arbitrary DNF formulas in the PAC model with membership queries, provided that the error is measured with respect to the uniform distribution. This extends results on learning decision trees, disjoint DNF and Satisfy j DNF formulas in the same model ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proceedings of the IEEE Symp. on Foundation of Computer Science, 1994. To Appear. 21

.... has been a central question in computational learning theory since it was posed by Valiant in 1984 [59] DNF formulas have recently been shown to be learnable in models such as the superset and subset query model [20] and the PAC model with membership queries under the uniform distribution [39]. However, it remains an open question to determine whether DNF formulas are learnable in polynomial time in two of the most widely studied and natural models: the PAC model, and the model of membership and equivalence queries. Recently, Bshouty gave a membership and extended equivalence query ....

J. Jackson, An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proc. of the 35th Annual Symposium on Foundations of Comp. Sci., pages 42--53, IEEE Computer Society Press, Los Alamitos, CA, 1994.

....n) under the uniform distribution. The impact of the Fourier transform on learning theory cannot be underestimated judging from the sequence of papers that followed the paper [LMN93] This technique alone has made some outstanding results possible in recent years culminating in Jackson s result [J94] on the PAC learnability of DNF formulas under the uniform distribution with membership queries. In this paper, we study monotone Boolean functions using harmonic analysis on the cube. Our main result is that any monotone Boolean function has most of its power spectrum on its Fourier coefficients ....

Jeffrey Jackson. An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 42--53, 1994.

....as hard as finding a (strong) PAC learning algorithm for DNF formulas under any distribution. Freund [F90,F92] showed that weak PAC learning a class under any distribution D that is poly away from D, i.e. satisfies D=poly(n) D Dpoly(n) implies PAC learning under the distribution D. Jackson [J94] showed that DNF is weakly learnable under any distribution that is poly away from the uniform distribution and then using Freund s result gave a PAC learning algorithm for DNF formulas under the uniform distribution that uses membership queries. Jackson showed that for every distribution that is ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proceeding of the 35th Annual Symposium on Foundations of Computer Science, 1994.

....theorem known as Occam s Razor would give a stronger result in the sense that the underlying distribution may be arbitrary (that is, not necessarily uniform) This however comes at a price of a linear, as opposed to logarithmic, dependence of the sample query complexity on n. 3 Sieve algorithm [Jac97] This algorithm has query complexity O r log 2 n 2 2 , where r is the number of variables appearing in the DNF formula, and is the number of terms. However, this algorithm does not output a DNF formula as its hypothesis. On the other hand, Angluin [Ang88] ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. JCSS, 55:414--440, 1997.

....of the input. Any function can be represented as a linear combination of the basis functions. Kushilevitz and Mansour [KM93] gave a general 1 technique to recover the significant coefficients. They showed that this is sufficient for learning decision trees under the uniform distribution. Jackson [J94] extended the result to learning DNF under the uniform distribution. The output hypothesis is a majority of parities. Also, Jackson [J95] generalizes his DNF learning algorithm from uniform distribution to any fixed constant bounded product distribution. The lattice based techniques are, at a ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proceeding of the 35th Annual Symposium on Foundations of Computer Science, 1994.

....Among them, propositional formulas have received particular attention. It is known that learning general propositional formulas is hard [3, 18] in the usual learning models, but some efficiently learnable subclasses of Boolean formulas, especially inside CNF and DNF, have been identified (see [1, 2, 5, 13] for example) First order logic is a formalism with superior expressive power, but it is not so well studied from the computational learning point of view (see [19] and further references in that paper) An active line of research in predicate logics is, for instance, Inductive Logic Programming ....

Jeffrey C. Jackson. An Efficient Membership-query Algorithm for Learning DNF with respect to the Uniform Distribution. Journal of Computer and System Sciences, 55(3):414--440, December 1997. 26

.... for a wide class of algorithms that includes the top down decision tree approach (and also all variants of this approach that have been proposed to date) 2] The positive results for efficient decision tree learning in computational learning theory all make extensive use of membership queries [11, 5, 4, 9], which provide the learning algorithm with black box access to the target function (experimentation) rather than only an oracle for random examples. Clearly, the need for membership queries severely limits the potential application of such algorithms, and they seem unlikely to encroach on the ....

J. Jackson. An efficient membership query algorithm for learning DNF with respect to the uniform distribution. In Proceedings of the 35th IEEE Symposium on the Foundations of Computer Science. IEEE Computer Society Press, Los Alamitos, CA, 1994.

....O(log n) term DNF under the uniform distribution (a somewhat more general result was given by Blum and Rudich [6] Mansour [23] gave a n O(log log n) time membership query algorithm which learns arbitrary polynomial size DNF under the uniform distribution. In a celebrated result, Jackson [15] gave a polynomial time membership query algorithm for learning polynomial size DNF under constant bounded product distributions. His algorithm, the efficiency of which was subsequently improved by several authors [8, 20] is the only known polynomial time algorithm for learning the unrestricted ....

....: Then m 2B 2 ffl 2 ln 2 ffi implies that Pr fi fi fi fi s m m Gamma p fi fi fi fi ffl ffi: 2. 1 The Learning Model Our learning model is a distribution specific version of Valiant s Probably Approximately Correct (PAC) model [26] and has been studied by many researchers, e.g. [3, 5, 8, 9, 10, 13, 15, 19, 21, 22, 23, 27, 28]. Let C be a class of Boolean functions over f0; 1g n ; let D be a probability distribution over f0; 1g n ; and let f 2 C be an unknown target function. A learning algorithm A for C takes as input an accuracy parameter 0 ffl 1 and a confidence parameter 0 ffi 1: During its execution ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution, J. Comput. Syst. Sci. 55 (1997), 414-440.

....in his seminal 1984 paper introducing the PAC learning model [36] more than fifteen years later this question is widely regarded as one of the most important open problems in learning theory. While many partial results have been given for restricted versions of the DNF learning problem (see e.g. [8, 9, 21, 23, 24, 26, 27, 32, 37, 38]) the difficulty of the unrestricted DNF learning problem is evidenced by the fact that, prior to the current work, only two algorithms were known which improve on the naive 2 n time bound [11, 35] The first subexponential time algorithm for learning DNF was due to Bshouty [11] who gave an ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. J. Comput. Syst. Sci. 55 (1997), 414-440.

....of DNF have been shown to be learnable not much progress has been made for the general case. Two recent results illustrate the state of the art in DNF learning. Bshouty et.al. 7] gave a randomized algorithm, using restricted subset and superset queries, to learn DNF. In the PAC model, Jackson [17] has given an algorithm using membership queries to learn DNF against the uniform distribution. We demonstrate the power of our approach by giving a deterministic teacher learner pair for the class of DNF formulas. The learner runs in polynomial time and uses only equivalence queries. Furthermore, ....

Jeffrey Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In 35th Annual Symposium on Foundations of Computer Science, pages 42--53, November 1994.

....n) under the uniform distribution. The impact of the Fourier transform on learning theory cannot be underestimated judging from the sequence of papers that followed the paper [LMN93] This technique alone has made some outstanding results possible in recent years culminating in Jackson s result [J94] on the PAC learnability of DNF formulas under the uniform distribution with membership queries. In this paper, we study monotone Boolean functions using harmonic analysis on the cube. Our main result is that any monotone Boolean function has most of its power spectrum on its Fourier coefficients ....

Jeffrey Jackson. An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages 42--53, 1994.

....proved that for constant dimension d, unions of non discretized rectangles over the d dimensional Euclidean space are pac learnable. Long and Warmuth proved that for constant k, unions of k non discretized rectangles over arbitrary dimensional Euclidean space are pac learnable. Recently, Jackson [J] proved that any union of polynomially many discretized rectangles over the domain [0; n Gamma 1] d such that each of those rectangles is bounded on O( log d log log n ) sides is strongly pac learnable with respect to the uniform distribution and using membership queries as well. In the ....

J. Jackson, "An efficient membership-query algorithm for learning DNF with respect to the uniform distribution", Proc of the 35th Annual Symposium on Foundations of Computer Science, 1994.

....to be computationally practical. In this section we show how to substantially improve the algorithm s time dependency on the error parameter ; thus making progress towards a more efficient implementation. 5.2. An Overview of the Harmonic Sieve Jackson proves the following theorem: Theorem 21 [19] The class of DNF formulae over f0; 1g n is strongly learnable under the uniform distribution using membership queries in time O(ns 8 = 12 ) where s is the DNF size of the target function f and is the accuracy parameter. The algorithm outputs as its final hypothesis a ....

....circuit. At the heart of Jackson s Harmonic Sieve algorithm is a procedure WDNF [1] which uses queries to MEM(f) as well as calls to the example oracle EX(f; D) for weakly learning DNF (see Appendix A for a more detailed description of the WDNF algorithm) Jackson proves the following: Lemma 22 [19] For any boolean function f of DNF size s over f0; 1g n and any distribution D; algorithm WDNF runs in time O(ns 6 (L1 (2 n D) 6 ) and outputs a parity function which is a (1=2 1 =s) approximator to f under D: Proof Sketch of Theorem 7: The Harmonic Sieve algorithm works by applying ....

[Article contains additional citation context not shown here]

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. Journal of Computer and System Sciences 55 (1997), pp. 414-440.

....4.3, we believe our circuit size parameter to be optimal with respect to this class of techniques. Table 1 summarizes our hard core set construction results. We also show how to use Impagliazzo s hard core set construction to obtain a new variant of Jackson s breakthrough Harmonic Sieve algorithm [17] for learning DNF formulae with membership queries under the uniform distribution. Our variant is substantially more efficient than the original algorithm. Jackson s original algorithm runs in time O(ns 8 = 12 ) where n is the number of variables in the DNF formula, s is the number of ....

....MEM(f) returns the value f(x) of the unknown target function on x: Another relaxation of the PAC model is to require that the learning algorithm succeed not for an arbitrary distribution but only under the uniform distribution. In a breakthrough result ten years after Valiant s paper, Jackson [17] gave an algorithm, the Harmonic Sieve, which uses membership queries to learn DNF in polynomial time under the uniform distribution. Although his algorithm runs in polynomial time, it is not considered to be computationally practical. In this section we show how to substantially improve the ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In "35th Annual Symposium on Foundations of Computer Science," (1994), pp. 42-53.

....algorithmic procedure; given a function and a threshold parameter it finds in polynomial time all the Fourier coefficients of the function that are greater than the threshold. Originally the procedure was used to learn decision trees [5] and later it was used for learning polynomial size DNF [8, 2, 4]. The Fourier transform technique applies naturally to the uniform distribution, and, indeed, most of the learnability results based on the Fourier transform are with respect to the uniform distribution. Some of the results were extended to product distribution [1, 3] A great advantage of the ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Annual Symposium on Switching and Automata Theory, pages 42 -- 53, 1994.

....were used in the literature for PAClearning decision tree with membership queries, the Fourier transform technique and the Lattice based techniques. Kushilevitz and Mansour [KM93] gave a technique for learning decision trees under the uniform distribution via the Fourier Spectrum. Jackson [J94] extended the result to learning DNF under the uniform distribution. The output hypothesis is a majority of parities. Jackson [J95] generalizes his DNF learning algorithm from uniform distribution to any fixed constant bounded product distribution. Definition 1 A product distribution is fixed ....

J. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proceeding of the 35th Annual Symposium on Foundations of Computer Science, 1994.

.... for a wide class of algorithms that includes the top down decision tree approach (and also all variants of this approach that have been proposed to date) 2] The positive results for efficient decision tree learning in computational learning theory all make extensive use of membership queries [14, 5, 4, 11], which provide the learning algorithm with black box access to the target function (experimentation) rather than only an oracle for random examples. Clearly, the need for membership queries severely limits the potential application of such algorithms, and they seem unlikely to encroach on the ....

J. Jackson. An efficient membership query algorithm for learning DNF with respect to the uniform distribution. In Proceedings of the 35th IEEE Symposium on the Foundations of Computer Science. IEEE Computer Society Press, Los Alamitos, CA, 1994.

....noise (a noise rate near 1 2 ) in the membership oracle. Second, we generalize work of Kushilevitz and Mansour [37] showing that a minor modification of their algorithm for learning PL 1 (defined below) learns PL 1 from a persistently noisy membership oracle. Earlier versions of this work [31, 32] contained an erroneous proof that our result could be extended to a strong noise tolerant algorithm for DNF. However, that proof overlooked the fact that the boosting distributions D i generated by F1 are not only dependent on the target function and weak hypotheses produced, but are also ....

J. Jackson, An efficient membership-query algorithm for learning DNF with respect to the uniform distribution, in Proceedings of the 35th Annual Symposium on Foundations of Computer Science, 1994, pp. 42--53.

....history. Valiant [18] introduced the problem and gave efficient algorithms for learning certain subclasses of DNF. Since then, learning algorithms have been developed for a number of other subclasses of DNF [13, 4, 3, 11, 2, 1, 7, 16, 9] and recently for the unrestricted class of DNF expressions [6, 12], but almost all of these results and in particular the results for the unrestricted class use membership queries (the learner is told the output value of the target function on learner specified inputs) This has left open the question of to what extent membership queries are necessary for ....

....noise, our result is evidence that quantum learning algorithms may be better able to handle noise than traditional algorithms. To obtain our quantum DNF learning algorithm, we modify the recent Harmonic Sieve algorithm (HS) for learning DNF with respect to uniform using membership queries [12]. In fact, HS properly learns the larger class d PT 1 of functions expressible as a threshold of a polynomial number of parity functions, and our algorithm properly learns this class as well. The Harmonic Sieve uses membership queries to locate parity functions that correlate well with the ....

[Article contains additional citation context not shown here]

Jackson, J. An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution. in: Proceedings of the 35th Annual Symposium on Foundations of Computer Science. 1994. To appear.

....yet another contrast, we also prove a hardness result showing, among other things, that distribution free learnability of even 1 DL requires access to at least half of the bits in each example. Our study of k TOP is motivated in part by the fact that it is known to have useful Fourier properties [Jac94]; furthermore, it has also been studied in the context of empirical machine learning [Jac95] We exploit the Fourier properties of k TOP to show first that k TOP is weakly k RFA learnable and that this learning is efficient for constant k. We then use our k DL results in conjunction with the weak ....

....6.1 Weak Learnability of k TOP Our first observation is that the class k TOP of thresholds of k parities is weakly learnable from a k RFA oracle, and the learning is efficient for constant k. This follows from the following lemma, which is a slight modification of a similar result in [Jac94]. Lemma 6.1 Let f be any k TOP of size s and D any distribution over the domain of f . Then there exists a parity a with jaj k such that j Pr x2D [f = a ] Gamma 1 2 j 1 2s : Because there are only n k k parities, and because the correlation of a k parity with target f can be ....

Jeffrey C. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proceedings of the 35th Annual IEEE Symposium on Foundations of Computer Science, pages 42--53, 1994.

....Christino Tamon Dept. Math Comp. Science Clarkson University Potsdam, NY 13699 5815, U.S.A. tino clarkson.edu Abstract An efficient algorithm exists for learning disjunctive normal form (DNF) expressions in the uniformdistribution PAC learning model with membership queries [J97], but in practice the algorithm can only be applied to small problems. We present several modifications to the algorithm that substantially improve its asymptotic efficiency. First, we show how to significantly improve the time and sample complexity of a key subprogram, resulting in similar ....

....the sample size required for PAC learning with membership queries under a fixed distribution and apply this technique to the uniform distribution DNF learning problem. Finally, we present a learning algorithm for DNF that is attribute efficient in its use of random bits. 1 INTRODUCTION Jackson [J97] gave the first polynomial time PAC learning algorithm for DNF with membership queries under the uniform distribution. However, the algorithm s time and sample complexity make it impractical for all but relatively small problems. The algorithm is also not particularly efficient in its use of ....

[Article contains additional citation context not shown here]

J. C. Jackson. An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution. Journal of Computer and System Sciences, 55(3):414-440, 1997.

No context found.

J. C. Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. Journal of Computer and System Sciences, 55:414--440, 1997.

No context found.

Jeffrey Jackson. An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. In Proceedings 35th Annual Symposium on Foundations of Computer Science, pages 42-53, Santa Fe, New Mexico, November 1994. IEEE.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC