G. Pagallo. Adaptive Decision Tree Algorithms for Learning from Examples. PhD thesis, University of California, Santa Cruz, 1990.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....trees method that solves classification exercises with higher accuracy in comparison with classical orthogonal decision trees. For binary decision trees, the methods, which construct new attributes as conjunctions, disjunctions or negations of basic attributes are useful. The Fringe method [6], for example, constructs new attributes directly from the branches of decision tree, using constraints from nodes. The CI method evaluates new attributes generated from decision tree [7] Classification problems have relatively simple graphical interpretation. The task of classification is ....

Pagallo. Adaptive Decision Tree Algorithms for Learning from Examples//in: Ph.D. Thesis, University of California at Santa Cruz. CA, 1990.

....fundamental limitation of selective induction algorithms is that when task supplied attributes are not adequate for describing theories to be learned, their performance in terms of prediction accuracy and theory complexity is usually poor. The replication problem (Pagallo and Haussler, 1990; Pagallo, 1990) of decision trees (Quinlan, 1993; Breiman, Friedman, Olshen and Stone, 1984) is a manifestation of this fundamental limitation of selective induction. Since a decision tree divides an instance space into mutually exclusive regions to represent a concept, a tree may contain duplication of a ....

....Most of these algorithms choose conditions for constructing new attributes from nodes that are near the fringe of a tree. They are referred to as the FRINGE family of algorithms. The FRINGE family of algorithms can solve the replication problem to some extent. FRINGE (Pagallo and Haussler, 1990; Pagallo, 1990) constructs a new attribute using the conjunction of two conditions at the parent and grandparent nodes of a positive leaf. Since conjunctions are created only from positive paths and the method of generating new attributes can only be applied to binary trees, FRINGE is limited to learning DNF ....

[Article contains additional citation context not shown here]

PAGALLO, G. (1990): Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D. Thesis, University of California at Santa Cruz, Santa Cruz, CA.

....deteriorates, yet real world problems such as protein folding exhibit millions of peaks [Rendell, 1988; Lathrop, Webster, and Smith, 1987] This is a manifestation of the fundamental limitation of selective induction. The replication problem of decision trees [Pagallo and Haussler, 1990; Pagallo, 1990] is another manifestation of the fundamental limitation of selective induction. Since a decision tree divides an instance space into mutually exclusive regions, to represent a concept, a tree may contain duplication of a sequence of tests in different paths such as that shown (grey parts) in ....

....their examples and theories (or concepts) Different types of attribute are appropriate for describing different characteristics of examples. Most selective induction algorithms can accept attributes of these three kinds. However, many existing constructive induction algorithms such as Fringe [Pagallo, 1990] and Citre [Matheus and Rendell, 1989] only construct new binary attributes by using logical operators such as conjunction, negation, and disjunction. The two research topics mentioned above explore two novel methods of constructing new attributes for decision tree learning, but they also generate ....

[Article contains additional citation context not shown here]

G. Pagallo, Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D. Thesis, University of California at Santa Cruz, Santa Cruz, CA.

....representing concepts. Note that attributes with more than two ordered discrete values can be specified as either nominal or numeric attributes. Most selective induction algorithms can accept attributes of these three kinds. However, many existing constructive induction algorithms such as Fringe (Pagallo, 1990), Citre (Matheus Rendell, 1989) CI (Zheng, 1992) LFC (Ragavan Rendell, 1993) and CAT (Zheng, 1998) only construct new binary attributes by using logical operators such as conjunction, negation, and disjunction. On the other hand, ID2 of 3 (Murphy Pazzani, 1991) creates at least M of N ....

....attributes in the form of X of N representations. During the generation of a tree, the construction of X of N attributes occurs. 3.1. Building decision trees Like ID2 of 3 (Murphy Pazzani, 1991) XofN consists of a single process while other constructive induction algorithms such as Fringe (Pagallo, 1990) and AQ17 hci (Wnek Michalski, 1994) interleave two processes, namely selective induction and new attribute construction. As shown in Table 1, XofN recursively builds a decision tree by constructing, at each decision node, one new nominal X of N attribute based on primitive attributes using the ....

[Article contains additional citation context not shown here]

Pagallo, G. (1990). Adaptive Decision Tree Algorithms for Learning from Examples. Doctoral dissertation, Department of Computer and Information Sciences, University of California, Santa Cruz, CA.

....an important role in constructive induction. However, all other factors may also affect the performance of a learning system. Each of them might, either positively or negatively, contribute to performance differences between different algorithms. Most constructive induction systems such as Fringe [Pagallo, 1990], LFC [Ragavan and Rendell, 1993] and AQ17 hci [Wnek and Michalski, 1994] use conjunction and or disjunction as constructive operators. That is, the constructed attributes are conjunctions or disjunctions of other attributes. A few systems use other constructive operators, for example, M of N ....

.... [Michalski, 1978; Bloedorn, Michalski, and Wnek , 1993] and X of N [Zheng, 1995a] A comparison of two decision tree learning systems showed that the one constructing conjunctions as new attributes outperformed that constructing disjunctions for DNF concepts, and vice versa for CNF concepts [Pagallo, 1990]. Murphy and Pazzani, 1991] demonstrated that constructing M of N representations can improve the performance of decision tree learning. Zheng [1995] shows that X of N representations are more representationally powerful than conjunctive, disjunctive, and M of N representations. Nevertheless, all ....

[Article contains additional citation context not shown here]

G. Pagallo, Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D. Thesis, University of California at Santa Cruz, Santa Cruz, CA.

....of conjuncts of a CNF target concept) as new attributes, and thus can produce a decision tree with higher accuracy and lower complexity. However, it does not perform well on some real world problems, especially when real valued primitive attributes are concerned. Pagallo and Haussler s FRINGE [3,4] constructs new attributes by conjoining the parent and grandparent nodes of all positive leaves in the decision tree. It is appropriate for learning DNF expressions. To deal with CNF problems as well, Pagallo implemented SymFringe [4] that constructs new attributes by using the conjunctions of ....

....attributes are concerned. Pagallo and Haussler s FRINGE [3,4] constructs new attributes by conjoining the parent and grandparent nodes of all positive leaves in the decision tree. It is appropriate for learning DNF expressions. To deal with CNF problems as well, Pagallo implemented SymFringe [4] that constructs new attributes by using the conjunctions of the parent and grandparent nodes of all positive and negative leaves in the decision tree. Yang et al. [7] present an algorithm DCFringe that also conjoins the parent and grandparent nodes of all positive leaves in the decision tree but ....

[Article contains additional citation context not shown here]

G. Pagallo, Adaptive Decision Tree Algorithms for Learning from Examples, PHD thesis, (University of California at Santa Cruz, 1990).

....with pruning [4] It considers all possible combinations of conditions in a path and is efficient in practice as some parts of the search space are eliminated during search. This search method is due to Webb s work on rule learning [4] 2 The CAT Algorithm Like the Fringe family of algorithms [2, 5, 6] and the CI algorithms [3] CAT is also a hypothesisdriven constructive induction algorithm for learning multivariate trees. It constructs new binary attributes by using the dynamic path based method over previously learned decision trees. Its constructive operators are conjunction and negation ....

....and is the number of attribute value pairs, or conditions, in the test of the node for a multivariate tree. Computational requirements will be briefly addressed in the final subsection. SFringe is a member of the Fringe family of hypothesis driven constructive decision tree learning algorithms [5]. It follows the idea of SymFringe [6] with a straightforward extension. For each leaf, it constructs one new attribute using the conjunction of two conditions at the parent and grandparent nodes of the leaf. SFringe adopts the fixed path based strategy. CI3 is also a hypothesis driven ....

[Article contains additional citation context not shown here]

G. Pagallo, Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D. Thesis, University of California at Santa Cruz, Santa Cruz, CA, 1990.

....fundamental limitation of selective induction algorithms is that when task supplied attributes are not adequate for describing theories to be learned, their performance in terms of prediction accuracy and theory complexity is usually poor. The replication problem (Pagallo and Haussler, 1990; Pagallo, 1990) of decision trees (Quinlan, 1993; Breiman, Friedman, Olshen, and Stone, 1984) is a manifestation of this fundamental limitation of selective induction. Since a decision tree divides an instance space into mutually exclusive regions, to represent a concept, a tree may contain duplication of a ....

....Most of these algorithms choose conditions for constructing new attributes from nodes that are near the fringe of a tree. They are referred to as the Fringe family of algorithms. The Fringe family of algorithms can solve the replication problem to some extent. Fringe (Pagallo and Haussler, 1990; Pagallo, 1990) constructs a new attribute using the conjunction of two conditions at the parent and grandparent nodes of a positive leaf. Since conjunctions are created only from positive paths and the method of generating new attributes can only be applied to binary trees, Fringe is limited to learning DNF ....

[Article contains additional citation context not shown here]

Pagallo, G. (1990): Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D.

....ones. Experimental results on a set of artificial and real world domains support these statements. 1. Introduction A wide variety of methods has been explored to construct new binary attributes by using, as constructive operators, logical AND, NOT, OR [Matheus and Rendell, 1989; Pagallo, 1990; Ragavan and Rendell, 1993] and M of N [Murphy and Pazzani, 1991] Some researchers construct new continuous values attributes by using attribute counting operators [Michalski, 1978; Bloedorn et al. 1993] or mathematical operators such as multiplication and division [Michalski, 1978; Langley et ....

....cpu time for C4.5, XofN, XofN(c) and XofN(cc) on the Cleveland Heart Disease domain is 0.2, 14.8, 16.8, and 34.0 seconds respectively. On the Tic Tac Toe domain, it is 0.2, 25.7, 49.9, and 64.4 seconds respectively. 4.1. Experimental domains and methods Two artificial domains used here are from [Pagallo, 1990]. One is a randomly generated DNF concept: DNF4. It has ten terms with 64 binary attributes. Twenty nine of them are irrelevant. The other is an even parity function with 5 relevant and 27 irrelevant attributes: Parity5. We use the same experimental method given in [Pagallo, 1990] For each ....

[Article contains additional citation context not shown here]

G. Pagallo, Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D. thesis, University of California at Santa Cruz, 1990.

....its search strategy. The model space searched by our system is the combined model classes of linear discriminant functions, decision trees and instance based classifiers. We choose these model classes because previous research has illustrated that each has different strengths (Schlimmer, 1987; Pagallo, 1990; Utgoff, 1989; Utgoff Brodley, 1990; Aha, 1990; Brodley Utgoff, 1992) Because the ease with which the different representation languages can describe a particular concept varies, combining them allows the system to learn good generalizations for a wider class of learning tasks (Utgoff, ....

....on only the initial Boolean input features. Several extensions to the original FRINGE algorithm have been implemented. Because a small Conjunctive Normal Form (CNF) concept may not have a small DNF representation, the dual FRINGE algorithm was created to generate features useful for CNF concepts (Pagallo, 1990). For each leaf labeled negative in the tree, dual FRINGE forms a new feature by taking the disjunction of the two tests immediately above the leaf. Symmetric FRINGE is the result of combining both the FRINGE and dual FRINGE algorithms. The behavior of dual FRINGE on CNF concepts was similar to ....

[Article contains additional citation context not shown here]

Pagallo, G. M. (1990). Adaptive decision tree algorithms for learning from examples. Doctoral dissertation, University of California at Santa Cruz.

....different new attribute evaluation functions, different new attribute search methods, and so on. Constructive operators play an important role in constructive induction. However, other factors also affect the performance of a learning system. Most constructive induction systems such as FRINGE [5] and LFC [7] use conjunction or disjunction as constructive operators. A few systems use other constructive operators, for example, M of N [4] attribute counting operators [2, 1] and X of N [8] To explore the effects on constructive induction, more specifically on decision tree learning, of ....

....accuracy and theory complexity. The theory complexity [8] is the modified tree size that includes both decision nodes and leaves, and takes into account the sizes of new attributes at decision nodes. 4.1. Experimental domains and methods Fourteen artificial (logical) domains are from Pagallo [5]. We use the same experimental method given by Pagallo [5] including the sizes of training and test sets. Experiments are repeated ten times in each of these domains. In addition to the fourteen artificial domains, ten real world domains from the UCI repository of machine learning databases [3] ....

[Article contains additional citation context not shown here]

G. Pagallo. Adaptive decision tree algorithms for learning from examples (Ph.D. thesis). Technical report, University of California at Santa Cruz, Santa Cruz, CA, 1990.

....algorithms can accept attributes of these three kinds. However, many existing 1 Some more sophisticated attributes such as structured attributes may be used, but here we talk about only these three most commonly used types of attributes. constructive induction algorithms such as FRINGE [ Pagallo, 1990 ] and CITRE [ Matheus and Rendell, 1989 ] construct new boolean attributes only by using logical operators such as , and . ID2 of 3 [ Murphy and Pazzani, 1991 ] creates, as new attributes, M of N representations stating whether at least M of N conditions are true. M of N representations are ....

....more like continuous valued attributes than nominal attributes. 13 The boolean counting attribute is a special case of the #VarEQ(x) attribute, while the #VarEQ(x) attribute is a special case of the X of N representation. Most hypothesis driven constructive induction algorithms such as FRINGE [ Pagallo, 1990 ] CITRE [ Matheus and Rendell, 1989 ] CI [ Zheng, 1992 ] and AQ17 HCI [ Wnek and Michalski, 1994 ] construct and select a set of new attributes based on the entire training set. This strategy has a shortcoming: new attributes that have high values of the evaluation function for the entire ....

G. Pagallo, Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D. thesis, University of California at Santa Cruz, 1990.

....per se. When one arrives at a virtually pruned decision node, one treats it as a leaf, returning the corresponding class label. One method for deciding whether a subtree is virtually pruned is to apply the minimum description length principle. This would lend itself to an incremental approach. Pagallo (1990) showed that one can find good candidate compound tests by examining the fringe of an existing decision tree. By considering such a compound test and building a new tree, a better fit of the data is often found. Unfortunately, the FRINGE algorithm calls for building a new tree from scratch in each ....

Pagallo, G. M. (1990). Adaptive decision tree algorithms for learning from examples. Doctoral dissertation, University of California at Santa Cruz.

....2 Figure 1: The left side of the figure shows a simple axis parallel tree that uses two attributes. The right side shows the partitioning that this tree creates in the attribute space. Researchers have also studied decision trees in which the test at a node uses boolean combinations of attributes (Pagallo, 1990; Pagallo Haussler, 1990; Sahami, 1993) and linear combinations of attributes (see Section 2) Different methods for measuring the goodness of decision tree nodes, as well as techniques for pruning a tree to reduce overfitting and increase accuracy have also been explored, and will be discussed ....

Pagallo, G. (1990). Adaptive Decision Tree Algorithms for Learning From Examples. Ph.D.

No context found.

G. Pagallo. Adaptive Decision Tree Algorithms for Learning from Examples. PhD thesis, University of California, Santa Cruz, 1990.

No context found.

Giulia M. Pagallo. Adaptive Decision Tree Algorithms for Learning from Examples. PhD thesis, University of California, Computer Research Laboratory, Santa Cruz, CA, June 1990.

No context found.

G. Pagallo, Adaptive Decision Tree Algorithms for Learning from Examples, Ph.D. Thesis, University of California at Santa Cruz, Santa Cruz, CA (1990).

No context found.

Pagallo, G. M. (1990). Adaptive decision tree algorithms for learning from examples. Doctoral dissertation, University of California at Santa Cruz.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC