Utgoff, P. E., & Brodley, C. E. (1990). An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning, Los Altos, CA.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....attained. If necessary, the intermediate process can also be explicitly explained by rules R i Gammah and R h Gammao . C4.5 and C4.5rules [12] were run on the above three datasets to generate DT rules. Briefly, C4.5 generates a decision tree which C4.5rules generalizes to rules. Since researchers [1, 17, 20] observed that mapping many valued variables to two valued variables results in decision trees with higher classification accuracy 3 , the same binary coded data for neural networks were used for C4.5 and C4.5rules. Being explicable is only one aspect of understandability. A rule with many ....

P.E. Utgoff and C.E. Brodley. An incremental method for finding multivariate splits for decision trees. In Machine Learning: Proceedings of the Seventh International Conference, pages 58--65. University of Texas, Austin, Texas, 1990.

....in a linear combination test Different solutions have been suggested. In CART, Breiman et al. [2] suggested to form a linear combination using only the ordered features. Another solution is to map each multi valued unordered feature to m numeric features, one for each observed value of the feature [26]. This is equivalent to the binary coding mentioned earlier when the neural net input coding was discussed. 5 Related Work In order to overcome the replication problem and alleviate the fragmentation problem, researchers have suggested various solutions. 16] proposed compound boolean features; ....

P.E. Utgoff and C.E. Brodley. An incremental method for finding multivariate splits for decision trees. In Machine Learning: Proceedings of the Seventh International Conference, pages 58--65. University of Texas, Austin, Texas, 1990.

....1) constraints and m Theta (k Gamma 1) k Theta (n 1) variables (not counting slacks) Since many such LPs may be needed to find a single decision tree, a fast method is desired. Ideally, an iterative method is also desirable in case new points are added and the tree needs to be adjusted [21]. Previous iterative approaches based on extensions to the perceptron algorithm [19, 6, 7] do not have stable performance for the inseparable case and as a result heuristic methods [12, 5] have been developed to get around this deficiency. Ideally we would like to have a fast parallelizable ....

P. E. Utgoff and C. E. Brodley. An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning, pages 58--65, Los Altos, CA, 1990. Morgan Kaufmann.

....sum of distances of the misclassified objects to it, using a convergent procedure for adjusting its coefficients. Perceptron trees, which are decision trees with perceptrons just above the leaf nodes, were discussed in [480] Decision trees with perceptrons at all internal nodes were described in [482, 438]. Mathematical Programming: Linear programming has been used for building adaptive classifiers since late 1960s [216] Given two possibly interesecting sets of points, Duda and Hart [117] proposed a linear programming formulation for finding the split whose distance from the misclassified points ....

....adaptive split depends on the training subsample it is splitting. An overly simple example of an adaptive split is a test on the mean value of a feature. Utgoff et al. proposed incremental tree induction methods in the context of univariate decision trees [479, 481] as well as multivariate trees [482]. Crawford [99] shows that approaches like Utgoff s, which attempt to update the tree so that the best split according to the 16 Smoothing is the process of adjusting probabilities at a node in the tree based on the probabilities at other nodes on the same path. Averaging improves probability ....

[Article contains additional citation context not shown here]

Paul E. Utgoff and Carla E. Brodley. An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning, pages 58--65, Los Altos, CA, 1990. Morgan Kaufmann.

....lead to difficult to characterize effects that may not be well approximated by methods with high inductive bias. There are a number of immediate enhancements that can be made to the learning algorithm. The first is to embed decision trees that use non axis parallel splits such as perceptron trees [22], or linear discriminants that might be more effective, since they will in general provide much better performance for shallow trees. Some simple branch and bound tests can be implemented to prevent unnecessary subdivision. In particular, if the node s current best case split has a p Gamma ....

P. Utgoff and C.E. Brodley. An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Workshop on Machine Learning, MorganKauffman, 1990.

....from the exponential family other than the Gaussian or the multinomial. They fit cleanly within the EM framework discussed in this paper; in particular, the IRLS algorithm is available for the M step of EM. A topic of recent interest in the decision tree literature (Murthy, Kasif Salzberg, 1993; Utgoff Brodley, 1990) involves the use of decision hyperplanes at oblique angles to the axes. As discussed earlier, oblique decision hyperplanes arise in the probabilistic approach as the parametric component of multinomial logit models. Our empirical results, as well as our theoretical convergence results, suggest ....

Utgoff, P. E., & Brodley, C. E. (1990). An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning, Los Altos, CA.

....respect to the decision tree methodology in the machine learning literature. Algorithms such as ID3 build trees that have axisparallel splits and use heuristic splitting algorithms (Quinlan, 1986) More recent research has studied decision trees with oblique splits (Murthy, Kasif Salzberg, 1993; Utgoff Brodley, 1990). None of these papers, however, have treated the problem of splitting data as a statistical problem, nor have they provided a global goodness of fit measure for their trees. There are a variety of neural network architectures that are related to the HME architecture. The multi resolution aspect ....

Utgoff, P. E., & Brodley, C. E. (1990). An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning, Los Altos, CA.

....has de emphasized computational issues in most applications. 1 In the active subarea of learning decision tree classifiers, examples of methods that improve accuracy are: ffl Construction of multi attribute tests using logical combinations (Ragavan and Rendell 1993) arithmetic combinations (Utgoff and Brodley 1990; 1 For extremely large datasets, however, learning time can remain the dominant issue (Catlett 1991; Chan and Stolfo 1995) Heath, Kasif, and Salzberg 1993) and counting operations (Murphy and Pazzani 1991; Zheng 1995) ffl Use of error correcting codes when there are more than two classes ....

Utgoff, P. E., and Brodley, C. E. 1990. An incremental method for finding multivariate splits for decision trees. In Proceedings 7th International Conference on Machine Learning, 58-65. San Francisco: Morgan Kaufmann.

....most other decision tree methods, which use single variable tests (e.g. ID3 [15] and CART [5] Geometrically speaking, oblique trees use oblique hyperplanes to partition the training set. Multivariate tree building algorithms have also been studied by Breiman et al. 5] and Utgoff and Brodley [23]. Both of our methods, SADT and OC1, are randomized algorithms, unlike previous decision tree methods. Due to this randomization, the algorithms produce different trees each time they are run. This turns out to be an advantage, since we can run the algorithm many times in an attempt to find the ....

P. Utgoff, and C. Brodley. An incremental method for finding multivariate splits for decision trees. Proceedings of the Seventh International Conference on Machine Learning, pp. 56--65. Los Altos, CA: Morgan Kaufmann, 1990.

....to other neural network algorithms such as back propagation. For example, the XOR network of figure 2 above was produced in a fraction of a second using roughly 30 epochs. Detailed results are presented in section 4.1. 3 Previous Work A hybrid system known as perceptron trees (Utgoff, 1988; Utgoff and Brodley, 1990) is the work most closely related to the ideas presented here. A perceptron tree is a decision tree, where each node of the tree is a linear threshold unit (perceptron) The inner nodes of the tree are used as decision nodes to split examples along different paths of the tree. Leaf nodes are ....

Utgoff, P. E. and Brodley, C. E. (1990). An incremental method for finding multivariate splits for decision trees. Proceedings of the Ninth National Conference on Artificial Intelligence, pages 58--65.

....sum of distances of the misclassified objects to it, using a convergent procedure for adjusting its coefficients. Perceptron trees, which are decision trees with perceptrons just above the leaf nodes, were discussed in [362] Decision trees with perceptrons at all internal nodes were described in [365, 334]. Mathematical Programming: Linear programming has been used for building adaptive classifiers since late 1960s [156] Given two possibly intersecting sets of points, Duda and Hart [85] proposed a linear programming formulation for finding the split whose distance from the misclassified points is ....

....split depends on the training subsample it is splitting. An overly simple example of an adaptive split is a test on the mean value of a feature. Utgoff et al. proposed incremental tree induction methods in the context of univariate decision trees [361, 363, 364] as well as multivariate trees [365]. Crawford [69] argues that approaches which attempt to update the tree so that the best split according to the updated sample is taken at each node, suffer from repeated restructuring. This occurs because the best split at a node vacillates widely while the sample at the node is still small. An ....

Paul E. Utgoff and Carla E. Brodley. An incremental method for finding multivariate splits for decision trees. In Proc. of the Seventh Int. Conf. on Machine Learning, pages 58--65, Los Altos, CA, 1990. Morgan Kaufmann.

....This is not the case in HMM modelization, since the contribution of all the speakers is used for the world model. Some improvements can still be added to the current system of binary classifiers. On the one hand, different learning algorithms can be used, like MLPs, oblique decision trees [9], or other well suited 2class separators. On the other hand, further attention can be given to the choice of the anti speaker set, given its important role, which is to describe in the acoustic parameter space the whole non speaker area. Another issue that can be very interesting in the speaker ....

P. E. Utgoff and C. E. Brodley. An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning, pages 58--65, Los Altos, CA, 1990. Morgan Kaufmann.

....of Omega might be considered as close if their are close according to the Euclidian distance. However, two points, even far from each other according to the Euclidian distance, can be considered as close according to networks based on perceptrons, or to decision trees with multivariate splits [UB90] if they are linearly separable from almost all other points in Omega Gamma 4.2 Elaboration of the decomposition In the present research a quite simple approach has been chosen, which does not depend on the learning method used for the dichotomies. Matrix D is constructed row by row in a ....

P. E. Utgoff and C. E. Brodley. An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning, pages 58--65, Los Altos, CA, 1990. Morgan Kaufmann.

....divides feature space so as to separate them as much as possible, and the multivariate decision tree recursively trains new LTUs on the two halves of feature space. The result, therefore, is a tree of linear threshold units, dividing feature space into arbitrary (multi dimensional) polygons (see [25, 9] for more detailed descriptions) 3.4 Learning Weights in the Linear Threshold Units. Several methods exist for learning tests in a linear threshold unit. Brodley and Utgoff [5] discuss four such methods: the Recursive Least Squares (RLS) 29] the Pocket al..gorithm [15] Thermal Training [14] ....

....is labeled as a class. A decision tree therefore consists of nodes that are either decisions or classes. The LTU is therefore a discriminant function in the form of a multivariate decision tree; the resulting tree is much more compact and efficient than a tree comprised of univariate decisions [25]. Figure 3: An outdoor scene comprising of brick buildings and a natural background 3.6 Performance Initial tests indicate that the multivariate decision trees are surprisingly accurate, even when the feature space is restricted to the RGB values of the pixels (after smoothing) Figure 4shows the ....

Utgoff, P.E., and Brodley, C.E., "An Incremental Method for Finding Multivariate Splits for Decision Trees", Proc. of the Seventh International Conference on Machine Learning, Austin, TX, 1990, Morgan-Kaufman.

.... of only single rather than sets of literals (i.e. it suffers from a horizon effect) Several extensions of ID3 (Quinlan, 1986) have attempted to correct this problem by providing look ahead capabilities, multi attribute splitting tests, or supporting constructive induction (e.g. Seshu, 1989; Utgoff Brodley, 1990; Matheus, 1990) Similar extensions could help FOIL.1 either by improving its information gain evaluation function directly or by providing it with information that allows it to extend clauses with several literals simultaneously. These two approaches have been implemented in CHAM and FOCL ....

Utgoff, P. E., & Brodley, C. E. (1990). An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning (pp. 58--65).

....form x a, where a is a value in the observed range of feature x. Because these tests are based on a single input variable, univariate trees can only divide feature space orthogonally to a feature s axis. This introduces a bias that may be inappropriate for problems with linearly related features[1, 11]. Utgoff and Brodley [11] overcome this problem by using the perceptron learning rule to induce decision trees in which the tests are linear combinations of features. Linear machine decision trees generalize the two category multivariate splits permitted by the perceptron training rule to ....

....in the observed range of feature x. Because these tests are based on a single input variable, univariate trees can only divide feature space orthogonally to a feature s axis. This introduces a bias that may be inappropriate for problems with linearly related features[1, 11] Utgoff and Brodley [11] overcome this problem by using the perceptron learning rule to induce decision trees in which the tests are linear combinations of features. Linear machine decision trees generalize the two category multivariate splits permitted by the perceptron training rule to n category multivariate splits by ....

Utgoff, P.E. and Brodley, C.E. "An Incremental Method for Finding Multivariate Splits for Decision Trees," Proc. of the Seventh International Conference on Machine Learning, Austin, TX., 1990. Morgan-Kaufman.

.... rule as described below, will find a solution machine in a finite number of steps (Duda Hart, 1973) A multivariate test, here represented as a linear machine, makes it possible to represent decision boundaries that are not orthogonal to the axes (Breiman, Friedman, Olshen Stone, 1984; Utgoff Brodley, 1990). Such a capability is essential if one wants to capture and represent a concept that is best described in terms of a function of two or more variables. For example, one would want to express the concept of an overweight person in terms of a relationship between height and weight. A linear ....

Utgoff, P. E., & Brodley, C. E. (1990). An incremental method for finding multivariate splits for decision trees. Proceedings of the Seventh International Conference on Machine Learning (pp. 58-65). Austin, TX: Morgan Kaufmann.

....in a linear combination test One solution, used in CART (Breiman, et al. 1984) is to form linear combinations using only the ordered features. An alternative solution is to map each unordered feature to m ordered features, one for each observed value of the feature (Hampson Volper, 1986; Utgoff Brodley, 1990). In order to map an unordered feature to a numeric feature, one needs to be careful not to impose an order on the values of the unordered feature. For a two valued feature, one can simply assign 1 to one value and Gamma1 to the other. If the feature has more than two observed values, then each ....

.... for a separating hyperplane because the classification accuracy of an LTU trained using the absolute error correction rule is unpredictable when the instances are not linearly separable (Duda Hart, 1973) The Pocket al..gorithm was used in PT2, an incremental multivariate decision tree algorithm (Utgoff Brodley, 1990). Phi Phi Phi Phi Phi Phi Phi Phi Phi Phi Phi Phi Phi Phi Phi Phi o o o o o o o x x x x x x x o Figure 2: Nonseparable Instance Space 3.3 The Thermal Training Procedure The thermal training procedure can be applied to a linear threshold unit or a linear machine. We discuss ....

Utgoff, P. E., & Brodley, C. E. (1990). An incremental method for finding multivariate splits for decision trees. Proceedings of the Seventh International Conference on Machine Learning (pp. 58-65). Austin, TX: Morgan Kaufmann.

No context found.

Utgoff, P. E., & Brodley, C. E. (1990). An incremental method for finding multivariate splits for decision trees. In Proceedings of the Seventh International Conference on Machine Learning, Los Altos, CA.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC