Nader Bshouty. Exact Learning via the Monotone Theory. In IEEE Foundations of Computer Science, pages 302--311, 1993.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....is monotone if it is monotone in all its input variables. It is unate if it is unate in all its input variables. Unate functions have been studied extensively in switching theory [6, 7] More recently, they have been exploited in the development of algorithms in computational learning theory [1, 3]. The class of monotone Boolean functions has a simple characterization in terms of forbidden projections. The class consists of exactly those functions that do not have any projections equivalent to g(x) x. In contrast, the characterization of unate functions by forbidden projections is ....

N. Bshouty, Exact Learning via the Monotone Theory, Proceedings of the 34th Annual IEEE Symposium on the Foundations of Computer Science (1993) 302--311.

....is as hard as the problem of learning general DNF. Among these learnable subclasses are: Monotone DNF [Val84, Ang88] Read twice DNF [Han91, AP91, PR95] Horn DNF (at most one literal negated in every term) AFP92] log n term DNF [BR92] and DNF CNF (this class includes decision trees) Bsh93] Read k DNF Particularly relevant to our work is the considerable attention that has been given to the learnability of boolean formulas where each variable occurs some bounded (often constant) number k of times ( read k ) Recently, polynomial time algorithms have been given for exactly ....

....DNF representation might have an exponentially long disjoint representation. However, every decision tree can be efficiently expressed as a disjoint DNF formula, by creating a term corresponding to the assignment of variables along each branch that leads to a leaf labeled 1 . Recently, Bshouty [Bsh93] has given an algorithm (using equivalence and membership queries) for learning boolean formulas in time polynomial in the maximum of their DNF and CNF representations. Since every decision tree has a DNF and CNF that have size polynomial in the size of the original tree, his algorithm may be ....

[Article contains additional citation context not shown here]

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the IEEE Symp. on Foundation of Computer Science, pages 302--311, Palo Alto, CA., 1993.

....polynomial size decision trees in time O(n(lgn) Kushilevitz and Mansour [273] gave a polynomial time PAC learning algorithm with membership queries for decision trees under the uniform distribution. Hancock [189] gave a polynomial time algorithm for PAC learning read k decision trees. Bshouty [54] showed that decision trees are learnable under the model of exact learning with membership queries and unrestricted 55 equivalence queries. Recently, agnostic PAC learning [13] and pruning [205] have been studied by the learnability theory community. In the context of ordered binary decision ....

NADER H. BSHOUTY. Exact learning via monotone theory. In Proceedings. 3Jth Annual Symposium on Foundations of Computer Science, pages 302-311, New York, NY, 1993. IEEE.

....equivalence query algorithm for learning DNF, in which the equivalence queries are allowed to be depth three formulas, not just DNFs. Its running time depends on the size of the smallest CNF formula equivalent to the target DNF, and thus the algorithm does not always run in polynomial time [19]. A useful way to classify the complexity of DNF formulas from the perspective of learnability is based on the maximumnumber of occurrences (or reads ) of the variables in the formulas. It is known that read once DNF formulas can be exactly learned in polynomial time with equivalence and ....

....class F . However, there are classes for which an exact learning algorithm using extended equivalence queries (and membership queries) has been developed, but no proper learning algorithm is known. Examples include decision trees, simple deterministic languages, and read k satisfy j DNF formulas [19, 38, 2]. Similarly, there are some classes that provably cannot be learned in polynomial time using hypotheses from the original class (without membership queries) yet for which learning algorithms with extended equivalence queries (and no membership queries) are known [6, 51, 15] The results in [51, ....

N. Bshouty, Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302--311, IEEE Computer Society Press, Los Alamitos, CA, November 1993.

....logarithmic depth decision trees in which each node is decided by a monomial can be exactly identified using equivalence and membership queries. Although it was known that this was possible for (ordinary) logarithmic depth decision trees by the results of Kushilevitz and Mansour [17] and Bshouty [7], it appears that this result could not have been derived using previous methods for the case in which each decision node is a monomial. 7 Conclusions and Open Problems We have shown that sparse polynomials over a field F defined on several boolean variables can be exactly identified using ....

Nader H. Bshouty. Exact learning via the monotone theory. In 34th Annual Symposium on Foundations of Computer Science, November 1993.

....by just using a UAV EQ log oracle and an MQ oracle. Finally, a corresponding model of UAV PAC Learning can be naturally defined. 3 RELATED WORK Within the exact learning model a number of interesting polynomial time algorithms have been presented to learn target classes such as decision trees [19], deterministic finite automata [2] Horn sentences [4] read once formulas [5, 18, 16] read twice DNF formulas [1] k term DNF formulas [13, 20] etc. For all of these classes it is known (using information theoretic arguments) that neither membership queries nor equivalence queries alone ....

....polynomial time in the UAV model using only the UAV EQ oracle, then C is exactly learnable in the standard model in polynomial time using only the EQ oracle. 5 LEARNING ORDERED DECISION TREES WITH A UAV MQ ORACLE Combining our results from the last section with Bshouty s decision tree algorithm [19], we can learn the class of decision trees in the UAV model using the UAV MQ, UAV EQ, and EV oracles. As discussed in Section 2, for the class of 14 Tree BuildTree(proj) 1 if UAV MQ(proj Delta Delta Delta) 6= then return tree(UAV MQ(proj Delta Delta Delta) nil; nil) 2 TL : ....

Nader H. Bshouty. Exact learning via the monotone theory. Inform. Comput. , 123(1):146--153, 1995.

....membership and equivalence queries. 46 Corollary 6 The class of boolean decision trees is learnable in polynomial time with extended equivalence and malicious membership queries. Proof: Lemma 9 shows that the class of boolean decision trees is polynomially closed under finite exceptions. In [11] it was shown that it is learnable in polynomial time using membership and extended equivalence queries. Corollary 7 The class of monotone DNF formulas with finite exceptions is learnable in polynomial time with equivalence and malicious membership queries. Proof: Corollary 4 shows that the ....

N. Bshouty. Exact learning via the monotone theory. In Proc. of the 34th Symposium on the Foundations of Comp. Sci., pages 302--311. IEEE Computer Society Press, Los Alamitos, CA, 1993.

....cryptographic assumptions. In fact, at the time of Kharitonov s result, it appeared possible that his results would soon be extended to DNF; our result shows otherwise. Due to the lack of positive results for unrestricted DNF, various restricted DNF classes have attracted considerable attention [4, 2, 8, 3, 1, 5, 14, 6]. We extend these results. In particular, it is known that the class of read k DNF (DNF in which every variable appears at most k times) is learnable in polynomial time using membership queries for k 2 [2, 8] but is as hard to learn in the distribution free PAC model as unrestricted DNF for k ....

Nader H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302-311, 1993.

....the uniform distribution. This result was strengthened by Blum and Rudich [7] who gave a polynomial time algorithm for exact learning O(log n) term DNF using membership and equivalence queries; several other polynomial time algorithms for O(log n) term DNF have since been given in this model [3, 9, 10, 25]. Mansour [27] gave a n O(log log n) time membership query algorithm which learns polynomial size DNF under the uniform distribution. In a celebrated result, Jackson [18] gave a polynomial time membership query algorithm for learning polynomial size DNF under constant bounded product ....

....DNF under the uniform distribution using hypotheses which are monotone k term DNF. This was improved by Sakai and Maruoka [29] who gave a polynomialtime algorithm for learning monotone O(log n) term DNF under the uniform distribution using hypotheses which are monotone O(log n) term DNF. In [9] Bshouty gave a polynomial time uniform distribution algorithm for learning a class which includes monotone O(log n) term DNF. Later Bshouty and Tamon [12] gave a polynomial time algorithm for learning a class which includes monotone O(log 2 n= log log n) 3 ) term DNF under constant bounded ....

N. Bshouty. Exact learning via the monotone theory. Information and Computation 123(1) (1995), 146-153.

....decision trees is efficiently learnable over U , given access to the membership oracle. Prior to their result, there had been no polynomial time learning results for the unrestricted standard class of decision trees (and no results at all for the extended class they study) However, Bshouty [Bsh93] subsequently gave an efficient (non Fourier) algorithm for learning standard decision trees with membership queries over any distribution. On a different front, Mansour [Man92] combined the above lemma with Theorem 4.4 to get a tight bound on the number of significant Fourier coefficients in ....

....decision trees, given by Ehrenfeucht and Haussler [EH89] runs in time n O(logn) However, as we mentioned in the previous chapter, Kushilevitz and Mansour have used Lemma 4. 9 to give a polynomial time algorithm for learning Boolean decision trees over U with membership queries, and Bshouty [Bsh93] has given an efficient algorithm for learning decision trees over any distribution, with membership queries. He has further shown that any Boolean function is learnable (with membership queries) in time polynomial in the larger of its DNF and CNF sizes. Finally, Khardon [Kha94] uses Fourier ....

Nader Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302--311, 1993.

....The specific question of whether or not there is a winning strategy (a learning algorithm) for the DNF concept class in some reasonable learning model has been especially well studied. While a number of algorithms have been developed for learning various subclasses of DNF in a variety of models [46, 34, 7, 3, 30, 4, 2, 11, 38, 18], many of the results for unrestricted DNF have been negative. Angluin, who introduced the model of exact learning with proper equivalence queries [5] models mentioned here are defined in the next section) showed that DNF is not efficiently learnable in this model [6] Subsequently, Angluin and ....

N. H. Bshouty, Exact learning via the monotone theory, in Proceedings of the 34th Annual Symposium on Foundations of Computer Science, 1993, pp. 302--311.

.... for a wide class of algorithms that includes the top down decision tree approach (and also all variants of this approach that have been proposed to date) 2] The positive results for efficient decision tree learning in computational learning theory all make extensive use of membership queries [11, 5, 4, 9], which provide the learning algorithm with black box access to the target function (experimentation) rather than only an oracle for random examples. Clearly, the need for membership queries severely limits the potential application of such algorithms, and they seem unlikely to encroach on the ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th IEEE Symposium on the Foundations of Computer Science, pages 302--311. IEEE Computer Society Press, Los Alamitos, CA, 1993.

....n fug) g: 1) As the following theorem shows characteristic models capture all the information about Horn expressions. 5 Theorem 2.1 ( KKS93] Let H be a Horn expression then H = closure(char(H) 2. 2 Monotone Theory and Characteristic Models The monotone theory was introduced by Bshouty [Bsh93], and was used for a theory for modelbased reasoning in [KR94] Definition 2.1 (Order) We denote by the usual partial order on the lattice f0; 1g n , the one induced by the order 0 1. That is, for x; y 2 f0; 1g n , x y if and only if 8i; x i y i . For an assignment b 2 f0; 1g n we ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the IEEE Symp. on Foundation of Computer Science, pages 302--311, Palo Alto, CA., 1993.

....using both membership and equivalence queries. In this model, in addition to the class of positive functions, there are a number of classes such as read once functions (see, e.g. 2, 9] which are learnable in polynomial time in n and the length of the formula expressing the function. Recently, [8] showed that any Boolean function is polynomially learnable either as DNF (disjunctive normal form) or CNF (conjunctive normal form) 2. Definitions and Basic Properties. Let f be a positive function of n variables. f is completely characterized by one of the sets minT (f) and maxF (f ) since ....

N. H. Bshouty, Exact learning via the monotone theory, IEEE Symposium on Foundation of Computer Science, 34 (1993), pp. 302--311.

....ask a query, a helpful teacher could provide an answer (or answers to a series of queries) that would allow the learner to relocate itself to a subtree containing successful computations. We return to this point in Section 5, when we show the power of such interaction using the work of Bshouty [6]. When we discuss the desirability of preventing collusion we beg the question of what constitutes collusion. Collusion is difficult to define formally. Unfortunately, in an interactive model of teaching, it is even more difficult to prevent. We devised several collusion prevention schemes (each ....

....the increased power of equivalence queries when answered in a helpful way. 5. Relationship to Monotone Theory One of the most interesting recent results in learning theory research is the development of the monotone theory, and its application to the learning of decision trees, by Bshouty [6]. Bshouty defines a complexity measure for concept classes called the monotone dimension, denoted Mdim(C) for concept class C. A set S of instances is a monotone basis for f 2 C (denoted M basis(ffg) if f can be represented as a CNF formula such that every clause in f is falsified by some ....

Nader H. Bshouty. Exact learning via the monotone theory. In 34th Annual Symposium on Foundations of Computer Science, November 1993.

....particular, this contrasts the negative result of Jackson and Tompkins [16] that the class of 1 decision lists is not teachable without trusted information, and the negative result of Anthony et. al [4] that linearly separable Boolean 15 functions are not efficiently teachable. In fact, Bshouty s [7] result that arbitrary decision trees are learnable with membership and equivalence queries implies that a much broader class than 1 decision lists is T L teachable with a polynomial time learner. Letting A in the proof of Theorem 2 be the halving algorithm [5, 18] we immediately get the ....

Nader H. Bshouty. Exact learning via the monotone theory. In 34th Annual Symposium on Foundations of Computer Science, November 1993.

....model based approach becomes feasible if one can make correct inferences when working with a small subset of models. Some results along this line have been obtained (Kautz, Kearns, and Selman, 1995; Khardon and Roth, 1996) In particular, previous work in (Khardon and Roth, 1996) using ideas from (Bshouty, 1995), identified a small set of models, called the set of characteristic models of W , that supports correct reasoning. We briefly describe some of the relevant results we need, culminating in Theorem 4.1 that identifies the models needed for the algorithm MBR to be correct and efficient. Definition ....

....we have that 1 b i 0. Definition 4.2 The monotone extension of f with respect to b is: M b (f) fx j x b z; for some z 2 fg: Definition 4.3 A set B is a basis for f if f = V b2B M b (f) B is a basis for a class of functions F if it is a basis for all the functions in F . It is known (Bshouty, 1995; Khardon and Roth, 1996) that there are classes of functions which have a small basis. In particular, the class of common queries defined above has a polynomial size basis B c , and the set BH k = fu 2 f0; 1g n j weight(u) n Gamma kg is a basis for the class of k quasi Horn functions. ....

Bshouty, N. H. 1995. Exact learning via the monotone theory. Information and Computation, 123(1):146--153.

.... for a wide class of algorithms that includes the top down decision tree approach (and also all variants of this approach that have been proposed to date) 2] The positive results for efficient decision tree learning in computational learning theory all make extensive use of membership queries [14, 5, 4, 11], which provide the learning algorithm with black box access to the target function (experimentation) rather than only an oracle for random examples. Clearly, the need for membership queries severely limits the potential application of such algorithms, and they seem unlikely to encroach on the ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th IEEE Symposium on the Foundations of Computer Science, pages 302--311. IEEE Computer Society Press, Los Alamitos, CA, 1993.

....has been studied before [1, 4] and it strictly includes the class of decision trees. An example in [4] shows that disjoint DNF strictly includes the class of DNF intersection CNF (which includes decision trees) so that our results are not implied by the recent learnability result for the latter [3]. Fourier Transform and Learnability Every function over the boolean cube f0; 1g n can be viewed as a vector with 2 n entries. Each entry is the value of the function on the input that corresponds to that entry. This description allows us to consider function classes as subspaces of the ....

N. Bshouty. Exact learning via the monotone theory. In Proc. 23rd Ann. IEEE Symp. on Foundations of Computer Science, pages 302--311, 1993.

....[Kha95] is that the complexity of the two translation problems is equivalent under polynomial reductions. While the complexity of these problem is an open problem [MR86, EG94, FK94, Kha95] we show that results from the theory of reasoning with models [KR94b] and computational learning theory [Bsh93] can be used to derive some positive results. In particular, we show that a closed form (although not in the form of functional dependencies) for a given relation can be found. Furthermore, we show that, given a set of dependencies, an approximate Armstrong relation can be found, where ....

.... implies (j= used between Boolean functions is equivalent to the connective subset or equal ( used for subsets of f0; 1g n . That is, f j= g if and only if f g. 2. 1 Theory We start by describing some results of the Monotone Theory of Boolean functions, introduced by Bshouty [Bsh93] and then use those to present the theory of reasoning with models, developed in [KR94b] All the proofs in this section are omitted; they can be found in [KR94b] 2.1.1 Monotone Theory Definition 2.1 (Order) We denote by the usual partial order on the lattice f0; 1g n , the one induced by ....

[Article contains additional citation context not shown here]

N. H. Bshouty. Exact learning via the monotone theory. In IEEE Symp. of Foundation of Computer Science, pages 302--311, Palo Alto, CA., 1993.

....size decision trees in time O(n O(logn) Kushilevitz and Mansour [273] gave a polynomial time PAC learning algorithm with membership queries for decision trees under the uniform distribution. Hancock [189] gave a polynomial time algorithm for PAC learning read k decision trees. Bshouty [54] showed that decision trees are learnable under the model of exact learning with membership queries and unrestricted equivalence queries. Recently, agnostic PAC learning [13] and pruning [205] have been studied by the learnability theory community. In the context of ordered binary decision ....

Nader H. Bshouty. Exact learning via monotone theory. In Proceedings. 34th Annual Symposium on Foundations of Computer Science, pages 302--311, New York, NY, 1993. IEEE.

....can show that it is possible to use a fairly small set of models as the Test Set, and still perform reasonably good inference. In the rest of this section we describe general conditions under which this can indeed be done. This section briefly introduces the monotone theory of Boolean functions [3], and the theory of reasoning with models 1 (see [11] for more details) An example to the notions introduced here is presented at the end of the section. 1 We note that this direction was studied independently in the Relational Data Base community [2, 16] The results on model based reasoning ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the IEEE Symp. on Foundation of Computer Science, pages 302--311, Palo Alto, CA., 1993.

....using membership and equivalence queries. Corollary 5 The class of boolean decision trees is learnable in polynomial time with extended equivalence and malicious membership queries. Proof: Lemma 3 shows that the class of boolean decision trees is polynomially closed under finite exceptions. In [6] it was shown that it is learnable in polynomial time using membership and extended equivalence queries. Corollary 6 The class of monotone DNF formulas with finite exceptions is learnable in polynomial time with equivalence and malicious membership queries. Proof: Corollary 3 shows that the class ....

N. Bshouty. Exact learning via the monotone theory. In Proc. of the 34th Symposium on the Foundations of Comp. Sci., pages 302--311. IEEE Computer Society Press, Los Alamitos, CA, 1993.

....is monotone if it is monotone in all its input variables. It is unate if it is unate in all its input variables. Unate functions have been studied extensively in switching theory [6, 7] More recently, they have been exploited in the development of algorithms in computational learning theory [1, 3]. The class of monotone Boolean functions has a simple characterization in terms of forbidden projections. The class consists of exactly those functions that do not have any projections equivalent to g(x) x. In contrast, the characterization of unate functions by forbidden projections is ....

N. Bshouty, Exact Learning via the Monotone Theory, Proceedings of the 34th Annual IEEE Symposium on the Foundations of Computer Science (1993) 302--311.

....an efficient and noise tolerant PAC L2R (MB L2R, resp. algorithm for the reasoning problem RQ(F) that uses the example oracle EX(D) RQD (f j ) resp. A richer class of functions can be learned when given access to membership queries, in addition to examples [Angluin, 1988; Blum et al. 1994; Bshouty, 1993] Many of these algorithms can be extended to work over f0; 1; g n . In particular, using the algorithms studied in [Bshouty , 1993] we have: Theorem 2 There exists an efficient PAC L2R algorithm that uses RQD (f j ) and MQ(f j ) for the reasoning problem RQ(F) where (i) F is the class of ....

....(RQD (f j ) resp. A richer class of functions can be learned when given access to membership queries, in addition to examples [Angluin, 1988; Blum et al. 1994; Bshouty, 1993] Many of these algorithms can be extended to work over f0; 1; g n . In particular, using the algorithms studied in [Bshouty , 1993] we have: Theorem 2 There exists an efficient PAC L2R algorithm that uses RQD (f j ) and MQ(f j ) for the reasoning problem RQ(F) where (i) F is the class of Decision Trees over f0; 1; g n . ii) F is the class of log nCNF DNF over f0; 1; g n . We have discussed a knowledge ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the IEEE Symp. on Foundation of Computer Science, pages 302--311, Palo Alto, CA., 1993.

....all the information about the closure, they also capture all the information about the Horn expression. Theorem 1 (Kautz et al. 1995; Dechter Pearl, 1992) Let H be a Horn expression then H = closure(char(H) 2. 2 Monotone Theory and Characteristic Models The monotone theory was introduced by Bshouty (1993), and was later used for a theory for model based reasoning (Khardon Roth, 1994) This section explores the relations between the monotone theory and characteristic models. Definition 1 (Order) We denote by the usual partial order on the lattice f0; 1g n , the one induced by the order 0 1. ....

Bshouty, N. H. (1993). Exact learning via the monotone theory. In Proceedings of the IEEE Symp. on Foundation of Computer Science, pp. 302--311 Palo Alto, CA.

....model based approach becomes feasible if one can make correct inferences when working with a small subset of models. Some results on this line have been obtained (Kautz, Kearns, and Selman, 1995; Khardon and Roth, 1994b) In particular, previous work in (Khardon and Roth, 1994b) using ideas from (Bshouty, 1995)) identified a small set of models, called the set of characteristic models of W , that supports correct reasoning. We briefly describe some of the relevant results we will need, culminating in Theorem 5.1 that identifies the models that are needed for the algorithm MBR to be correct and ....

....we have that 1 b i 0. Definition 5.2 The monotone extension of f with respect to b is: M b (f) fx j x b z; for some z 2 fg: Definition 5.3 A set B is a basis for f if f = V b2B M b (f) B is a basis for a class of functions F if it is a basis for all the functions in F . It is known (Bshouty, 1995; Khardon and Roth, 1994b) that there are classes of functions which share the same small basis. In particular, the class of common queries defined above has a polynomial size basis B c , and the set BH k = fu 2 f0; 1g n j weight(u) n Gamma kg is a basis for the class of k quasi Horn ....

Bshouty, N. H. 1995. Exact learning via the monotone theory. Information and Computation, 123(1):146--153.

.... for a wide class of algorithms that includes the top down decision tree approach (and also all variants of this approach that have been proposed to date) 2] The positive results for efficient decision tree learning in computational learning theory all make extensive use of membership queries [13, 5, 4, 11], which provide the learning algorithm with black box access to the target function (experimentation) rather than only an oracle for random examples. Clearly, the need for membership queries severely limits the potential application of such algorithms, and they seem unlikely to encroach on the ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th IEEE Symposium on the Foundations of Computer Science, pages 302--311. IEEE Computer Society Press, Los Alamitos, CA, 1993.

....algorithms, when represented as a traditional (formula based) knowledge base. Second, we develop a L2R algorithm for the class of functions with polynomial size DNF, for which a L2C algorithm is not known. The results in this section use two recent results, one on learning via monotone theory [Bsh93] and the other on reasoning with models [KR94c] we first introduce some definitions and results from there. 6.1 Monotone Theory Definition 6.1 (Order) We denote by the usual partial order on the lattice f0; 1g n , the one induced by the order 0 1. That is, for x; y 2 f0; 1g n , x y if ....

....Membership Query Oracle for the reasoning problem (CNF DNF;Q) where Q is any class of relevant and common queries. Proof: The Learning to Reason algorithm learns a model based representation for the target function f and then uses model based reasoning to answer queries with respect to it. In [Bsh93] an algorithm is developed that uses an Equivalence Query oracle and a Membership Query Oracle to learn an exact representation of any function f 2 CNF DNF . As a byproduct of this algorithm, the set of all minimal models with respect to a basis B of f is produced. Using this set of models ....

[Article contains additional citation context not shown here]

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the IEEE Symp. on Foundation of Computer Science, pages 302--311, Palo Alto, CA., 1993.

No context found.

Nader Bshouty. Exact Learning via the Monotone Theory. In IEEE Foundations of Computer Science, pages 302--311, 1993.

No context found.

N. Bshouty. Exact learning via the monotone theory. Information and Computation, 123(1):146--153, 1995.

No context found.

Nader Bshouty. Exact Learning via the Monotone Theory. In Proceedings of 34-th Annual Symposium on Foundations of Computer Science, pp. 302--311, 1993.

No context found.

N. Bshouty. Exact Learning via the Monotone Theory. In FOCS, 1991.

No context found.

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993. 11

....of functions C, we define DT (X; Y; C) to be the class of all decision trees over X whose leaves are functions from C over Y . We study the learnability of DT (X; Y; C) using membership and equivalence queries. Boolean decision trees, DT (X; f0; 1g) were shown to be exactly learnable in [Bs93] but does this imply the learnability of decision trees that have non boolean leaves A simple encoding of all possible leaf values will work provided that the size of C is reasonable. Our investigation involves several cases where simple encoding is not feasible, i.e. when jCj is large. We ....

....learnable concepts belonging to a class C, DT (X; Y; C) when the separation between the variables X and Y is known. A simple algorithm for decision trees whose leaves are constants, DT (X; C) is also presented. Each case above requires at least s separate executions of the algorithm from [Bs93] where s is the number of distinct leaves of the tree but we show that if C is a bounded lattice, DT (X; C) is learnable using only one execution of this algorithm. 1 Introduction Rooted binary trees, or decision trees, provide a natural representation both visually and conceptually for ....

[Article contains additional citation context not shown here]

Nader H. Bshouty. Exact Learning via the Monotone Theory. In Proceeding of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993.

....(Note that these latter algorithms do not use any oracles in addition to their weak subset and weak superset queries. Next we consider learning problems where the learner can make membership queries (but no equivalence queries) and has access to an NP oracle. We show that (in the terminology of [B93]) the number of membership queries required to learn a boolean function is bounded by a polynomial in its monotone dimension, dual monotone dimension, DNF size, and CNF size. Two corollaries of this are that with an NP oracle and membership queries: monotone functions are learnable in time ....

....c(f) size CNF (f) to be the minimum number of clauses in a CNF that represents f . These two size measures are polynomially related to the standard size measure of the number of bits required to represent a boolean function in a DNF or a CNF form. Let C be a concept class over f0; 1g . In [B93], a measure called the monotone dimension m(C) and its dual (called the dual monotone dimension m (C) are defined. It was proven that there is an algorithm to learn any f 2 C that uses d(f)m(C) equivalence queries and n d(f)m(C) membership queries. More importantly any hypothesis h issued ....

[Article contains additional citation context not shown here]

Nader Bshouty. Exact Learning via the Monotone Theory. In IEEE Foundations of Computer Science, pages 302--311, 1993.

....that for every distribution that is poly away from the uniform distribution there is a boolean function g that can be found in polynomial time that agrees well with f . Then g can be used for the weak PAC learning. The technique used by Jackson was the Fourier transform approach for learning. In [Bs93] we used a different approach, the monotone theory, to show that any DNF is exactly learnable from membership and equivalence queries in time polynomial in the DNF and CNF size of the the target function. This implies PAC learnability of CDNF formulas (poly size DNF and CNF) and decision trees ....

....in the DNF size, the number of variables, 1=ffl and 1=ffi. In particular, the class of decision trees is strongly PAC learnable with membership queires under any distribution in parallel in polylogarithmic time with a polynomial number of processors. Our algorithm uses the monotone theory [Bs93]. The sequential version of the algorithm in [Bs93] cannot be parallelized because many of the queries asked in the algorithm rely on the answer of the previous one. In this paper we develop a new version of the algorithm and show that it can be easily changed to a parallel algorithm. Our parallel ....

[Article contains additional citation context not shown here]

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993. 16

....of polynomial size Disjunctive Normal Form expressions. The DNF learning problem has a long history. Valiant [32] introduced the problem and gave ecient algorithms for learning certain subclasses of DNF. Since then, learning algorithms have been developed for a number of other subclasses of DNF [25, 4, 2, 21, 3, 1, 11, 27, 13, 10] and recently for the unrestricted class of DNF expressions [22] but almost all of these results and in particular the results for the unrestricted class use membership queries (the learner is told the output value of the target function on learner speci ed inputs) While Angluin and Kharitonov ....

N. H. Bshouty, Exact learning via the monotone theory, in Proceedings of the 34th Annual Symposium on Foundations of Computer Science, 1993, pp. 302-311.

....necessary to arrive at a response and the leaves represent the responses themselves. In the recent past, considerable effort has been devoted to finding learning algorithms for decision trees [Bl92,EH89,H92a,H92b,H93,KM91,R87] Boolean decision trees were shown to be learnable by Bshouty [Bs93] using a technique called the monotone theory, which is based on Angluin s algorithm for learning monotone DNF formulas [A88] Using this result we investigate some situations where the leaves of the decision trees are non boolean. We now give a general definition for decision trees having ....

....completion of these questions classifies the problem to the point where an appropriate set of instructions or calculations may be applied. We are interested in learning these types of functions but we must first show that a boolean encoding coupled with an application of the algorithm of Bshouty [Bs93] is not always practical for this type of problem. Consider the problem of learning decision trees with leaves that are constant values, DT (X; C) One approach would be to encode the set C in a boolean space such as C f0; 1g m where jCj 2 m . This effectively reforms our problem as m ....

[Article contains additional citation context not shown here]

Nader H. Bshouty. Exact Learning via the Monotone Theory. In Proceeding of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993.

....alone. Note that these latter algorithms do not use any oracles in addition to their subset and superset queries. Next we consider learning problems where the learner can make membership queries (but no equivalence queries) and has access to an NP oracle. We show that (in the terminology of [B93]) the number of membership queries required to learn a boolean function is bounded by a polynomial in its monotone dimension, dual monotone dimension, DNF size, and CNF 2 size. We then present a learning algorithm that uses an NP oracle and membership queries to learn any boolean function in time ....

....Definition: Let f be a boolean function over f0; 1g n . Then the d(f) size DNF (f) is the minimum number of terms in a DNF that represents f . Similarly we define c(f) size CNF (f) to be the minimum number of clauses in a CNF that represents f . Let C be a concept class over f0; 1g n . In [B93], a measure called the monotone dimension m(C) and its dual (called the dual monotone dimension m (C) are defined. It was proven that there is an algorithm to learn any f 2 C that uses d(f)m(C) equivalence queries and n 2 d(f)m(C) membership queries. More importantly any hypothesis h ....

[Article contains additional citation context not shown here]

Nader Bshouty. Exact Learning via the Monotone Theory. In IEEE Foundations of Computer Science, pages 302--311, 1993.

....of polynomial size Disjunctive Normal Form expressions. The DNF learning problem has a long history. Valiant [18] introduced the problem and gave efficient algorithms for learning certain subclasses of DNF. Since then, learning algorithms have been developed for a number of other subclasses of DNF [13, 4, 3, 11, 2, 1, 7, 16, 9] and recently for the unrestricted class of DNF expressions [6, 12] but almost all of these results and in particular the results for the unrestricted class use membership queries (the learner is told the output value of the target function on learner specified inputs) This has left open the ....

Bshouty, N. H. Exact Learning via the Monotone Theory. in: Proceedings of the 34th Annual Symposium on Foundations of Computer Science. 1993, pp. 302--311.

....uniform distribution to any fixed constant bounded product distribution. Definition 1 A product distribution is fixed constant bounded if there is a constant 0 c 1=2, that is independent of the number of variables n, such that for any variable x i , c P rob[x i = 1] 1 Gamma c. Bshouty [Bs93] gave a technique for learning decision trees under any distribution via the Monotone Theory. Schapire and Sellie [SS93] gave a Lattice based algorithm for learning multivariate polynomials over the binary field under any distribution. In the former the output hypothesis for the decision tree is ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993.

....(Note that these latter algorithms do not use any oracles in addition to their weak subset and weak superset queries. Next we consider learning problems where the learner can make membership queries (but no equivalence queries) and has access to an NP oracle. We show that (in the terminology of [B93]) the number of membership queries required to learn a boolean function is bounded by a polynomial in its monotone dimension, dual monotone dimension, DNF size, and CNF size. Two corollaries of this are that with an NP oracle and membership queries: monotone functions are learnable in time ....

....c(f) size CNF (f) to be the minimum number of clauses in a CNF that represents f . These two size measures are polynomially related to the standard size measure of the number of bits required to represent a boolean function in a DNF or a CNF form. Let C be a concept class over f0; 1g n . In [B93], a measure called the monotone dimension m(C) and its dual (called the dual monotone dimension m (C) are defined. It was proven that there is an algorithm to learn any f 2 C that uses d(f)m(C) equivalence queries and n 2 d(f)m(C) membership queries. More importantly any hypothesis h ....

[Article contains additional citation context not shown here]

Nader Bshouty. Exact Learning via the Monotone Theory. In IEEE Foundations of Computer Science, pages 302--311, 1993.

....with membership and equivalence queries for the class of unate DNF with the addition of constant number of any other terms and the addition of O(log n) nonunate variables to the terms. The hypotheses to the equivalence queries are polynomial size DNF. This result is an improvement of the result in [Bs93] for learning almost monotone DNF, i.e. monotone DNF with the addition of constant number of nonmonotone terms. In [Bs93] the hypotheses to the equivalence queries are depth 3 formula. ffl A deterministic polynomial time learning algorithm with membership and equivalence queries for the class of ....

....and the addition of O(log n) nonunate variables to the terms. The hypotheses to the equivalence queries are polynomial size DNF. This result is an improvement of the result in [Bs93] for learning almost monotone DNF, i.e. monotone DNF with the addition of constant number of nonmonotone terms. In [Bs93] the hypotheses to the equivalence queries are depth 3 formula. ffl A deterministic polynomial time learning algorithm with membership and equivalence queries for the class of O(log n) term DNF. The hypotheses to the equivalence queries are polynomial size DNF. ffl A polynomial time learning ....

[Article contains additional citation context not shown here]

N. H. Bshouty. Exact learning via the monotone theory. In Proceeding of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993.

....choose polynomial number of assignments a i . If f(a i ) is zero for all the assignments then with probability close to 1 we have f j 0. For decision tree of depth O(log n) we can deterministicly zero test by choosing the assignments to be the (n; O(log n) universal set. For details see [Bs93]. We now show how to reduce zero test to learning. Let f 0 = f . Since we can zero test we can find the minimal i 1 such that f 0 j x 1 0; x i 1 0 j 0. This implies that f x 1 0; x i 1 Gamma1 0 = x i 1 f 1 (x i 1 1 ; x n ) for some multivariate polynomial f 1 2 MULF ....

N. H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Symposium on Foundations of Computer Science. pages 302--311, November 1993.

No context found.

Nader H. Bshouty. Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302--311, 1993.

No context found.

N. Bshouty. Exact learning via the monotone theory. FOCS 1993.

No context found.

N. Bshouty. Exact learning via the monotone theory. FOCS 1993.

No context found.

N. H. Bshouty. Exact learning via the monotone theory, in Proceedings of the 34th Annual Symposium on Foundations of Computer Science (1993) 302-311.

No context found.

N. Bshouty, "Exact learning via the monotone theory", Proc of the 35th Annual Symposium on Foundations of Computer Science, 1993.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC