Hancock, T., Jiang, T., Li, M. and Tromp, J. (1996). Lower bounds on learning decision lists and trees. Information and Computation 126, 114--122.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....also efficient: when K = M n,k , so that the class DL(K) is k DL, then the algorithm is easily seen to have running time O(mn k 1 ) for fixed k. There is no guarantee, however, that the algorithm will necessarily produce a decision list that is nearly as short as it could be, as Hancock et al. [12] have shown. Eiter et al. 8] considered 1 decision lists in some detail and were able to find an improved (that is, faster) extension algorithm. In fact, rather than a running time of O(mn ) their algorithm has linear running time O(mn) They also develop a polynomial delay algorithm for ....

T. Hancock, T. Jiang, M. Li and J. Tromp. Lower bounds on learning decision lists and trees. Information and Computation, 126(2), 1996: 114--122.

.... Information Processing Letters 66(1998) 165 170 Information Procssing Letters On the hardness of approximating the minimum consistent acyclic DFA and decision diagram Shinichi Shimozono a, Kouichi Hirata a. 1, Ayumi Shinohara b, 2 a Department qfArtificial Intelligence, Krushu Institute q 7 chnology. liz. uka 820 8502, ....

.... Information Processing Letters 66(1998) 165 170 Information Procssing Letters On the hardness of approximating the minimum consistent acyclic DFA and decision diagram Shinichi Shimozono a, Kouichi Hirata a. 1, Ayumi Shinohara b, 2 a Department qfArtificial Intelligence, Krushu Institute q 7 chnology. liz. uka 820 8502, ....

[Article contains additional citation context not shown here]

T. Hancock, T Jiang, M. Li, J. Tromp, Lower bounds on learning decision lists and trees, in: Proc. 12th Annual Symposium on Theoretical Aspects of Computer Science, Springer, Berlin, 1995, pp, 527-538.

.... in reverse lexicographic order if (A[S] NONCONSTANT) then set m : 1; else set m : 0; set q[S] 1 max m, max S(i)# 0,1 q S S(i) # S# 0,1 n q[S] Again, if f is not given as a full truth table, then deciding whether C(f) k for some k is NP complete [12]. 3.3. Degree as a Polynomial. Let deg(f) be the minimum degree of an n variate real multilinear polynomial p such that, for all X , p(X) f(X) Degree was introduced to query complexity by Nisan and Szegedy [17] who observed the relationship deg(f) D(f ) Later Beals et al. 5] ....

T. Hancock, T. Jiang, M. Li, and J. Tromp, Lower bounds on learning decision lists and trees, Information and Computation, 126, pp. 114--122, 1996.

....as in [10] yields the decision tree p(T; T ) which is decision equivalent to the much smaller decision tree T . In terms of induction the most natural question is to nd a smallest decision tree that is consistent with a given training set. This question has been proven to be NP hard in [6], while nding a smallest one with respect to the expected number of tests was already proven to be NP hard in [7] We want to stress that apart from inductive inference the concept of decision trees is of interest too for describing discrete functions, and in this latter setting our question of ....

Hancock, T., Jiang, T., Li, M., and Tromp, J. Lower bounds on learning decision lists and trees. Information and Computation 126 (1996), 114-122.

....It is also deduced in [1] that nding a minimum size DNF formula is NP hard for various size measures (e.g. the number of literals in the formula) 1] is an unpublished manuscript by Angluin For decision trees, which can be viewed as a form of DNF formulas, only one result is known. In [6] Hancock et al. show that the hA; Bi version of minimum node decision tree is NP complete. For general circuits the situation is even worse no NP completeness results are known. Although no NP completeness results are known for general circuits, Kearns Valiant have shown [11] that the hA; ....

....of the literature accounts for the survey part. With respect to presentation and extension we will mainly concentrate on DNF formulas. Hopefully, this should come as no surprise considering the heavily suggestive title of the thesis. Although we concentrate on DNF formulas, the result from [6] on the NP completeness of the hA; Bi version of minimum node decision tree is presented. We also observe that this result holds for two other measures on decision trees; the external path length measure and the depth measure. The author has taken the liberty to have references in quotations ....

[Article contains additional citation context not shown here]

Hancock, T., Jiang, T., Li, M., and Tromp, J., \Lower Bounds on Learning Decision Lists and Trees", in: Information and Computation 126 (1996) 114-122.

....most k literals and k is a constant, are probably approximately correct (PAC) learnable in Valiant s model [39] This has largely extended the classes of Boolean functions which are known to be learnable. In the sequel, decision lists have been studied extensively in the learning field, see e.g. [19, 8, 17, 9]. However, while it is known that decision lists generalize some classes of Boolean functions [34] their relationships to other classes such as Horn functions, read once functions, threshold functions, or 2monotonic functions, which are widely used in the literature, were only partially known ....

T. Hancock, T. Jiang, M. Li, and J. Tromp. Lower Bounds on Learning Decision Lists and Trees. Information and Computation, 126:114--122, 1996. Extended Abstract STACS '95.

....due to Li and Vazirani [15] They showed that, unless P=NP, the minimum consistent DFA cannot be approximated within the ratio 9 8 in polynomial time. Pitt and Warmuth [18] improved to the ratio opt k , where opt is the minimum number of states and k is any positive integer. Hancock et al. [14] investigated the minimum consistent problem for decision lists and decision trees that represent n ary Boolean functions. They showed that decision lists cannot be approximated in polynomial time within a factor of n c for some constant c 0, unless P=NP. They also showed that decision trees ....

....Consistent OBDD is NP hard, even either the set of positive examples or the set of negative examples consists of only one string. Proof. This can be shown by a log space reduction involving the translation of instances from Minimum Cover [11] to sets P; N of strings presented by Hancock et al. [14]. Minimum Cover is, given a collection C of subsets over U , to find a subcollection C 0 C that covers U and is as small as possible. Given an instance C, construct P; N as follows: i) For every i 2 U , P includes the string which is obtained by placing 1s on 0 n at all positions k for c k ....

[Article contains additional citation context not shown here]

Hancock, T., Jiang, T., Li, M. and Tromp, J.: Lower bounds on learning decision lists and trees, draft, 1996.

....algorithm for learning decision lists (sometimes called 1 decision lists) with k alternations. It outputs a decision list with k alternations depending on O(r k log k m) variables, where m is the size of the sample. Using recent non approximability techniques, Hancock, Jiang, Li, and Tromp [HJLT94] have shown that, unless NP ae DTIME[2 poly(log n) decision lists with k alternations cannot be approximated within a multiplicative factor of log k n and decision lists with an unbounded number of alternations cannot be approximated in polynomial time within a multiplicative factor of 2 ....

....to PAC learn k alternation decision lists depending on r variables by k alternation decision lists depending on O(r k log k (r=ffl) variables. The sample complexity of the PAC algorithm is O( 1 ffl (log 1 ffi r k log n log k ( r ffl ) Recently, Hancock, Jiang, Li, and Tromp [HJLT94] proved various hardness results for the problem of approximating the smallest consistent decision list. They showed that k alternation decision lists over n variables cannot be approximated within a multiplicative factor of log k n unless NP ae DTIME[2 poly(log n) Our Occam algorithm ....

T. Hancock, T. Jiang, M. Li, and J. Tromp. Lower Bounds on Learning Decision Lists and Trees, unpublished manuscript, 1994.

.... 2 Learning decision trees of minimum rank Ehrenfeucht and Haussler [6] showed that the function class represented by rankbounded binary decision trees is learnable in the sense of the basic PAC model [22] Its superclass functions determined by arbitrary binary decision trees is not learnable [9]. There is only one natural way to extend the original definition of the rank to general multivalued decision trees without losing the learnability property. Definition 1. The rank of a decision tree T , denoted by r(T ) is defined as: 1. If T consists of a single leaf, then r(T ) 0. 2. Else ....

Hancock, T., Jiang, T., Li, M., Tromp, J.: Lower bounds on learning decision lists and trees. Inf. Comput. 126 (1996) 114--122

.... that if c represents a decision list of p terms over n variables, then jcj = O(np) The class of k decision lists (each monomial contains at most k literals) is PAC learnable (Rivest, 1987) Decision lists are not known to be PAC learnable, but lower bounds on learning decision lists are given in (Hancock et al. 1996). We prove in this Section that the concept class of decision lists is PAC learnable under helpful distributions. Note that decision lists are a superset of DNF formulas and thus DNF formulas are learnable using decision lists. It can be proved, using a greedy heuristic, that DNF formulas are ....

Hancock, T., Jiang, T., Li, M., and Tromp, J. (1996). Lower bounds on learning decision lists and trees. Information and Computation, 126(2):114--122.

.... such that if c represents a decision list of p terms over n variables, then jcj = O(np) The class of k decision lists (each monomial contains at most k litterals) are PAC learnable ( 18] Decision lists are not known to be PAC learnable, but lower bounds on learning decision lists are given in [9]. The class of k decision lists is learnable in the teaching model of [8] Therefore it is probably exactly learnable under helpful distributions. We prove in this Section that the concept class of decision lists is PAC learnable under helpful distributions. Note that decision lists are a superset ....

T. Hancock, T. Jiang, M. Li and J. Tromp, Lower Bounds on Learning Decision Lists and Trees, Inform. and Computation 126 (1996) 114-122.

....here a non proper PAC learning algorithm that learns decision lists of length d within polynomial time w.r.t. n, 2 d , 1= and log 1= by using only O i 1= i log 1= 16 d log n(d log log n) 2 examples. Here we remark that, for proper learning, Hancock, Jiang, Li, and Tromp [5] recently proved that the class of decision lists of length at most d is not e ciently PAC learnable by the class of decision lists of length at most d2 log ffi d , for any 1, unless NP DTIME[2 polylog(n) The key observation here is the following combinatorial lemma: For obtaining ....

T. Hancock, T. Jiang, M. Li, and J. Tromp. Lower Bounds on Learning Decision Lists and Trees. In Proceedings of the 12th Annual Symposium on Theoretical Aspects of Computer Science, Lecture Notes in Computer Science, 527-538, 1995.

....most k literals and k is a constant, are probably approximately correct (PAC) learnable in Valiant s model [35] This has largely extended the classes of Boolean functions which are known to be learnable. In the sequel, decision lists have been studied extensively in the learning field, see e.g. [18, 8, 16, 9]. However, while it is known that decision lists generalize some classes of Boolean functions [31] their relationships to other classes such as Horn functions, read once functions, threshold functions, or 2 monotonic functions, which are widely used in the literature, were only partially known ....

T. Hancock, T. Jiang, M. Li, and J. Tromp. Lower Bounds on Learning Decision Lists and Trees. Information and Computation, 126:114--122, 1996. Extended Abstract STACS '95.

....to ours. Since we show that k term monotone decision lists are representable as k decision lists, it follows that, for any constant k, k term monotone decision lists are improperly PAC learnable as k decision lists using Rivest s algorithm [25] A slight modification of the negative results in [16, 24] shows that proper PAC learnability of this class without membership queries is not possible, unless RP = NP . Improper exact learning algorithms which use both membership and equivalence queries can also be obtained by using the techniques of Bshouty [10] or Kushilevitz [19] to show that O(log ....

....definition. Corollary 20 The class of k term monotone decision lists, k 2, is not learnable with equivalence queries alone. Proof. Follows from Theorem 19 and Angluin s theorem [2] on the approximate fingerprints property. 4. 3 PAC learning, Improper Learning, and Related Results A paper [16] by Hancock, Jiang, Li, and Tromp shows that k term decision lists cannot be properly PAC learned (without membership queries) unless RP = NP. The proof of this result can be altered slightly to show that k term monotone decision lists cannot be properly PAC learned either, under the same ....

T. Hancock, T. Jiang, M. Li, and J. Tromp. "Lower Bounds on Learning Decision Lists and Trees." Information and Computation, Vol. 126, No. 2, 1996, pp. 114-122.

....achive a sample complexity similar to Winnow for the same problem but our hypothesis size is more efficient with respect to the number of relevant variables. Winnow s hypothesis for this case is a threshold of n variables. Here we remark that, for proper learning, Hancock, Jiang, Li, and Tromp [11] recently proved that the class of decision lists of length at most d is not efficiently PAC learnable by the class of decision lists of length at most d2 log ffi d , for any ffi 1, unless NP DT IME[2 polylog(n) The key observation here is the following combinatorial lemma: For ....

T. Hancock, T. Jiang, M. Li, and J. Tromp. Lower Bounds on Learning Decision Lists and Trees. In Proceedings of the 12th Annual Symposium on Theoretical Aspects of Computer Science, Lecture Notes in Computer Science, 527-538, 1995.

....pac sense from labeled examples usually requires finding the approximately smallest decision tree with high probability. For some time, it has been open that whether or not the decision version of computing the smallest decision tree is NP complete. Recently, the following results were obtained in [8]. 1. The decision version of computing the minimum decision tree is NP complete. 2. In general a target decision tree cannot be approximated in polynomial time by decision trees using data consisting of labeled examples to within a factor of 2 log ffi n for ffi 1, unless NP is contained in ....

T. Hancock, T. Jiang, M. Li, and J. Tromp, Lower bounds for learning decision lists and trees. Proc. 12th Annual Symp. Theoret. Aspects of Comput. Sci. (STACS'95), Lecture Notes in Computer Science, Springer-Verlag, Heidelberg, 1995, 527-538.

No context found.

Hancock, T., Jiang, T., Li, M. and Tromp, J. (1996). Lower bounds on learning decision lists and trees. Information and Computation 126, 114--122.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC