Nader Bshouty. Exact Learning via the Monotone Theory. In IEEE Foundations of Computer Science, pages 302--311, 1993.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....is monotone if it is monotone in all its input variables. It is unate if it is unate in all its input variables. Unate functions have been studied extensively in switching theory [6, 7] More recently, they have been exploited in the development of algorithms in computational learning theory [1, 3]. The class of monotone Boolean functions has a simple characterization in terms of forbidden projections. The class consists of exactly those functions that do not have any projections equivalent to g(x) x. In contrast, the characterization of unate functions by forbidden projections is ....

N. Bshouty, Exact Learning via the Monotone Theory, Proceedings of the 34th Annual IEEE Symposium on the Foundations of Computer Science (1993) 302--311.

....is as hard as the problem of learning general DNF. Among these learnable subclasses are: Monotone DNF [Val84, Ang88] Read twice DNF [Han91, AP91, PR95] Horn DNF (at most one literal negated in every term) AFP92] log n term DNF [BR92] and DNF CNF (this class includes decision trees) Bsh93] Read k DNF Particularly relevant to our work is the considerable attention that has been given to the learnability of boolean formulas where each variable occurs some bounded (often constant) number k of times ( read k ) Recently, polynomial time algorithms have been given for exactly ....

....DNF representation might have an exponentially long disjoint representation. However, every decision tree can be efficiently expressed as a disjoint DNF formula, by creating a term corresponding to the assignment of variables along each branch that leads to a leaf labeled 1 . Recently, Bshouty [Bsh93] has given an algorithm (using equivalence and membership queries) for learning boolean formulas in time polynomial in the maximum of their DNF and CNF representations. Since every decision tree has a DNF and CNF that have size polynomial in the size of the original tree, his algorithm may be ....

[Article contains additional citation context not shown here]

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the IEEE Symp. on Foundation of Computer Science, pages 302--311, Palo Alto, CA., 1993.

....polynomial size decision trees in time O(n(lgn) Kushilevitz and Mansour [273] gave a polynomial time PAC learning algorithm with membership queries for decision trees under the uniform distribution. Hancock [189] gave a polynomial time algorithm for PAC learning read k decision trees. Bshouty [54] showed that decision trees are learnable under the model of exact learning with membership queries and unrestricted 55 equivalence queries. Recently, agnostic PAC learning [13] and pruning [205] have been studied by the learnability theory community. In the context of ordered binary decision ....

NADER H. BSHOUTY. Exact learning via monotone theory. In Proceedings. 3Jth Annual Symposium on Foundations of Computer Science, pages 302-311, New York, NY, 1993. IEEE.

....equivalence query algorithm for learning DNF, in which the equivalence queries are allowed to be depth three formulas, not just DNFs. Its running time depends on the size of the smallest CNF formula equivalent to the target DNF, and thus the algorithm does not always run in polynomial time [19]. A useful way to classify the complexity of DNF formulas from the perspective of learnability is based on the maximumnumber of occurrences (or reads ) of the variables in the formulas. It is known that read once DNF formulas can be exactly learned in polynomial time with equivalence and ....

....class F . However, there are classes for which an exact learning algorithm using extended equivalence queries (and membership queries) has been developed, but no proper learning algorithm is known. Examples include decision trees, simple deterministic languages, and read k satisfy j DNF formulas [19, 38, 2]. Similarly, there are some classes that provably cannot be learned in polynomial time using hypotheses from the original class (without membership queries) yet for which learning algorithms with extended equivalence queries (and no membership queries) are known [6, 51, 15] The results in [51, ....

N. Bshouty, Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302--311, IEEE Computer Society Press, Los Alamitos, CA, November 1993.

....logarithmic depth decision trees in which each node is decided by a monomial can be exactly identified using equivalence and membership queries. Although it was known that this was possible for (ordinary) logarithmic depth decision trees by the results of Kushilevitz and Mansour [17] and Bshouty [7], it appears that this result could not have been derived using previous methods for the case in which each decision node is a monomial. 7 Conclusions and Open Problems We have shown that sparse polynomials over a field F defined on several boolean variables can be exactly identified using ....

Nader H. Bshouty. Exact learning via the monotone theory. In 34th Annual Symposium on Foundations of Computer Science, November 1993.

....by just using a UAV EQ log oracle and an MQ oracle. Finally, a corresponding model of UAV PAC Learning can be naturally defined. 3 RELATED WORK Within the exact learning model a number of interesting polynomial time algorithms have been presented to learn target classes such as decision trees [19], deterministic finite automata [2] Horn sentences [4] read once formulas [5, 18, 16] read twice DNF formulas [1] k term DNF formulas [13, 20] etc. For all of these classes it is known (using information theoretic arguments) that neither membership queries nor equivalence queries alone ....

....polynomial time in the UAV model using only the UAV EQ oracle, then C is exactly learnable in the standard model in polynomial time using only the EQ oracle. 5 LEARNING ORDERED DECISION TREES WITH A UAV MQ ORACLE Combining our results from the last section with Bshouty s decision tree algorithm [19], we can learn the class of decision trees in the UAV model using the UAV MQ, UAV EQ, and EV oracles. As discussed in Section 2, for the class of 14 Tree BuildTree(proj) 1 if UAV MQ(proj Delta Delta Delta) 6= then return tree(UAV MQ(proj Delta Delta Delta) nil; nil) 2 TL : ....

Nader H. Bshouty. Exact learning via the monotone theory. Inform. Comput. , 123(1):146--153, 1995.

....membership and equivalence queries. 46 Corollary 6 The class of boolean decision trees is learnable in polynomial time with extended equivalence and malicious membership queries. Proof: Lemma 9 shows that the class of boolean decision trees is polynomially closed under finite exceptions. In [11] it was shown that it is learnable in polynomial time using membership and extended equivalence queries. Corollary 7 The class of monotone DNF formulas with finite exceptions is learnable in polynomial time with equivalence and malicious membership queries. Proof: Corollary 4 shows that the ....

N. Bshouty. Exact learning via the monotone theory. In Proc. of the 34th Symposium on the Foundations of Comp. Sci., pages 302--311. IEEE Computer Society Press, Los Alamitos, CA, 1993.

....cryptographic assumptions. In fact, at the time of Kharitonov s result, it appeared possible that his results would soon be extended to DNF; our result shows otherwise. Due to the lack of positive results for unrestricted DNF, various restricted DNF classes have attracted considerable attention [4, 2, 8, 3, 1, 5, 14, 6]. We extend these results. In particular, it is known that the class of read k DNF (DNF in which every variable appears at most k times) is learnable in polynomial time using membership queries for k 2 [2, 8] but is as hard to learn in the distribution free PAC model as unrestricted DNF for k ....

Nader H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302-311, 1993.

....the uniform distribution. This result was strengthened by Blum and Rudich [7] who gave a polynomial time algorithm for exact learning O(log n) term DNF using membership and equivalence queries; several other polynomial time algorithms for O(log n) term DNF have since been given in this model [3, 9, 10, 25]. Mansour [27] gave a n O(log log n) time membership query algorithm which learns polynomial size DNF under the uniform distribution. In a celebrated result, Jackson [18] gave a polynomial time membership query algorithm for learning polynomial size DNF under constant bounded product ....

....DNF under the uniform distribution using hypotheses which are monotone k term DNF. This was improved by Sakai and Maruoka [29] who gave a polynomialtime algorithm for learning monotone O(log n) term DNF under the uniform distribution using hypotheses which are monotone O(log n) term DNF. In [9] Bshouty gave a polynomial time uniform distribution algorithm for learning a class which includes monotone O(log n) term DNF. Later Bshouty and Tamon [12] gave a polynomial time algorithm for learning a class which includes monotone O(log 2 n= log log n) 3 ) term DNF under constant bounded ....

N. Bshouty. Exact learning via the monotone theory. Information and Computation 123(1) (1995), 146-153.

....decision trees is efficiently learnable over U , given access to the membership oracle. Prior to their result, there had been no polynomial time learning results for the unrestricted standard class of decision trees (and no results at all for the extended class they study) However, Bshouty [Bsh93] subsequently gave an efficient (non Fourier) algorithm for learning standard decision trees with membership queries over any distribution. On a different front, Mansour [Man92] combined the above lemma with Theorem 4.4 to get a tight bound on the number of significant Fourier coefficients in ....

....decision trees, given by Ehrenfeucht and Haussler [EH89] runs in time n O(logn) However, as we mentioned in the previous chapter, Kushilevitz and Mansour have used Lemma 4. 9 to give a polynomial time algorithm for learning Boolean decision trees over U with membership queries, and Bshouty [Bsh93] has given an efficient algorithm for learning decision trees over any distribution, with membership queries. He has further shown that any Boolean function is learnable (with membership queries) in time polynomial in the larger of its DNF and CNF sizes. Finally, Khardon [Kha94] uses Fourier ....

Nader Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302--311, 1993.

....The specific question of whether or not there is a winning strategy (a learning algorithm) for the DNF concept class in some reasonable learning model has been especially well studied. While a number of algorithms have been developed for learning various subclasses of DNF in a variety of models [46, 34, 7, 3, 30, 4, 2, 11, 38, 18], many of the results for unrestricted DNF have been negative. Angluin, who introduced the model of exact learning with proper equivalence queries [5] models mentioned here are defined in the next section) showed that DNF is not efficiently learnable in this model [6] Subsequently, Angluin and ....

N. H. Bshouty, Exact learning via the monotone theory, in Proceedings of the 34th Annual Symposium on Foundations of Computer Science, 1993, pp. 302--311.

.... for a wide class of algorithms that includes the top down decision tree approach (and also all variants of this approach that have been proposed to date) 2] The positive results for efficient decision tree learning in computational learning theory all make extensive use of membership queries [11, 5, 4, 9], which provide the learning algorithm with black box access to the target function (experimentation) rather than only an oracle for random examples. Clearly, the need for membership queries severely limits the potential application of such algorithms, and they seem unlikely to encroach on the ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th IEEE Symposium on the Foundations of Computer Science, pages 302--311. IEEE Computer Society Press, Los Alamitos, CA, 1993.

No context found.

Nader Bshouty. Exact Learning via the Monotone Theory. In IEEE Foundations of Computer Science, pages 302--311, 1993.

No context found.

N. Bshouty. Exact learning via the monotone theory. Information and Computation, 123(1):146--153, 1995.

No context found.

Nader Bshouty. Exact Learning via the Monotone Theory. In Proceedings of 34-th Annual Symposium on Foundations of Computer Science, pp. 302--311, 1993.

No context found.

N. Bshouty. Exact Learning via the Monotone Theory. In FOCS, 1991.

No context found.

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993. 11

....of functions C, we define DT (X; Y; C) to be the class of all decision trees over X whose leaves are functions from C over Y . We study the learnability of DT (X; Y; C) using membership and equivalence queries. Boolean decision trees, DT (X; f0; 1g) were shown to be exactly learnable in [Bs93] but does this imply the learnability of decision trees that have non boolean leaves A simple encoding of all possible leaf values will work provided that the size of C is reasonable. Our investigation involves several cases where simple encoding is not feasible, i.e. when jCj is large. We ....

....learnable concepts belonging to a class C, DT (X; Y; C) when the separation between the variables X and Y is known. A simple algorithm for decision trees whose leaves are constants, DT (X; C) is also presented. Each case above requires at least s separate executions of the algorithm from [Bs93] where s is the number of distinct leaves of the tree but we show that if C is a bounded lattice, DT (X; C) is learnable using only one execution of this algorithm. 1 Introduction Rooted binary trees, or decision trees, provide a natural representation both visually and conceptually for ....

[Article contains additional citation context not shown here]

Nader H. Bshouty. Exact Learning via the Monotone Theory. In Proceeding of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993.

....(Note that these latter algorithms do not use any oracles in addition to their weak subset and weak superset queries. Next we consider learning problems where the learner can make membership queries (but no equivalence queries) and has access to an NP oracle. We show that (in the terminology of [B93]) the number of membership queries required to learn a boolean function is bounded by a polynomial in its monotone dimension, dual monotone dimension, DNF size, and CNF size. Two corollaries of this are that with an NP oracle and membership queries: monotone functions are learnable in time ....

....c(f) size CNF (f) to be the minimum number of clauses in a CNF that represents f . These two size measures are polynomially related to the standard size measure of the number of bits required to represent a boolean function in a DNF or a CNF form. Let C be a concept class over f0; 1g . In [B93], a measure called the monotone dimension m(C) and its dual (called the dual monotone dimension m (C) are defined. It was proven that there is an algorithm to learn any f 2 C that uses d(f)m(C) equivalence queries and n d(f)m(C) membership queries. More importantly any hypothesis h issued ....

[Article contains additional citation context not shown here]

Nader Bshouty. Exact Learning via the Monotone Theory. In IEEE Foundations of Computer Science, pages 302--311, 1993.

....that for every distribution that is poly away from the uniform distribution there is a boolean function g that can be found in polynomial time that agrees well with f . Then g can be used for the weak PAC learning. The technique used by Jackson was the Fourier transform approach for learning. In [Bs93] we used a different approach, the monotone theory, to show that any DNF is exactly learnable from membership and equivalence queries in time polynomial in the DNF and CNF size of the the target function. This implies PAC learnability of CDNF formulas (poly size DNF and CNF) and decision trees ....

....in the DNF size, the number of variables, 1=ffl and 1=ffi. In particular, the class of decision trees is strongly PAC learnable with membership queires under any distribution in parallel in polylogarithmic time with a polynomial number of processors. Our algorithm uses the monotone theory [Bs93]. The sequential version of the algorithm in [Bs93] cannot be parallelized because many of the queries asked in the algorithm rely on the answer of the previous one. In this paper we develop a new version of the algorithm and show that it can be easily changed to a parallel algorithm. Our parallel ....

[Article contains additional citation context not shown here]

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993. 16

....of polynomial size Disjunctive Normal Form expressions. The DNF learning problem has a long history. Valiant [32] introduced the problem and gave ecient algorithms for learning certain subclasses of DNF. Since then, learning algorithms have been developed for a number of other subclasses of DNF [25, 4, 2, 21, 3, 1, 11, 27, 13, 10] and recently for the unrestricted class of DNF expressions [22] but almost all of these results and in particular the results for the unrestricted class use membership queries (the learner is told the output value of the target function on learner speci ed inputs) While Angluin and Kharitonov ....

N. H. Bshouty, Exact learning via the monotone theory, in Proceedings of the 34th Annual Symposium on Foundations of Computer Science, 1993, pp. 302-311.

No context found.

Nader H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302--311, 1993.

No context found.

N. Bshouty. Exact learning via the monotone theory. FOCS 1993.

No context found.

N. Bshouty. Exact learning via the monotone theory. FOCS 1993.

No context found.

N. H. Bshouty. Exact learning via the monotone theory, in Proceedings of the 34th Annual Symposium on Foundations of Computer Science (1993) 302-311.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC