Results 1  10
of
25
Omnivariate Decision Trees
"... Univariate decision trees at each decision node consider the value of only one feature leading to axisaligned splits. In a linear multivariate decision tree, each decision node divides the input space into two with a hyperplane. In a nonlinear multivariate tree, a multilayer perceptron at each node ..."
Abstract

Cited by 15 (8 self)
 Add to MetaCart
Univariate decision trees at each decision node consider the value of only one feature leading to axisaligned splits. In a linear multivariate decision tree, each decision node divides the input space into two with a hyperplane. In a nonlinear multivariate tree, a multilayer perceptron at each node divides the input space arbitrarily, at the expense of increased complexity and higher risk of overfitting. We propose omnivariate trees where the decision node may be univariate, linear, or nonlinear depending on the outcome of comparative statistical tests on accuracy thus matching automatically the complexity of the node with the subproblem defined by the data reaching that node. Such an architecture frees the designer from choosing the appropriate node type, doing model selection automatically at each node. Our simulation results indicate that such a decision tree induction method generalizes better than trees with the same types of nodes everywhere and induces small trees.
CostConscious Comparison of Supervised Learning Algorithms over Multiple Data Sets
"... We propose Multi2Test for ordering multiple learning algorithms on multiple data sets from “best ” to “worst. ” Our goodness measure uses a prior cost term additional to generalization error. Our simulations show that Multi2Test generates orderings using pairwise tests on error and different types o ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
We propose Multi2Test for ordering multiple learning algorithms on multiple data sets from “best ” to “worst. ” Our goodness measure uses a prior cost term additional to generalization error. Our simulations show that Multi2Test generates orderings using pairwise tests on error and different types of cost. 2 1.
Soft Decision Trees
"... We discuss a novel decision tree architecture with soft decisions at the internal nodes where we choose both children with probabilities given by a sigmoid gating function. Our algorithm is incremental where new nodes are added when needed and parameters are learned using gradientdescent. We visual ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
(Show Context)
We discuss a novel decision tree architecture with soft decisions at the internal nodes where we choose both children with probabilities given by a sigmoid gating function. Our algorithm is incremental where new nodes are added when needed and parameters are learned using gradientdescent. We visualize the soft tree fit on a toy data set and then compare it with the canonical, hard decision tree over ten regression and classification data sets. Our proposed model has significantly higher accuracy using fewer nodes. 1.
Design and Analysis of Classifier Learning Experiments in Bioinformatics: Survey and Case
"... Abstract—In many bioinformatics applications, it is important to assess and compare the performances of algorithms trained from data, to be able to draw conclusions unaffected by chance and are therefore significant. Both the design of such experiments and the analysis of the resulting data using st ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
Abstract—In many bioinformatics applications, it is important to assess and compare the performances of algorithms trained from data, to be able to draw conclusions unaffected by chance and are therefore significant. Both the design of such experiments and the analysis of the resulting data using statistical tests should be done carefully for the results to carry significance. In this paper, we first review the performance measures used in classification, the basics of experiment design and statistical tests. We then give the results of our survey over 1,500 papers published in the last two years in three bioinformatics journals (including this one). Although the basics of experiment design are well understood, such as resampling instead of using a single training set and the use of different performance metrics instead of error, only 21 percent of the papers use any statistical test for comparison. In the third part, we analyze four different scenarios which we encounter frequently in the bioinformatics literature, discussing the proper statistical methodology as well as showing an example case study for each. With the supplementary software, we hope that the guidelines we discuss will play an important role in future studies. Index Terms—Statistical tests, classification, model selection Ç
INCREMENTAL CONSTRUCTION OF COSTCONSCIOUS ENSEMBLES USING MULTIPLE LEARNERS AND REPRESENTATIONS IN MACHINE LEARNING
, 2009
"... ..."
Model Selection in Omnivariate Decision Trees
"... Abstract. We propose an omnivariate decision tree architecture which contains univariate, multivariate linear or nonlinear nodes, matching the complexityofthenodetothecomplexityofthedatareachingthatnode. We compare the use of different model selection techniques including AIC, BIC, and CV to choose ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract. We propose an omnivariate decision tree architecture which contains univariate, multivariate linear or nonlinear nodes, matching the complexityofthenodetothecomplexityofthedatareachingthatnode. We compare the use of different model selection techniques including AIC, BIC, and CV to choose between the three types of nodes on standard datasets from the UCI repository and see that such omnivariate trees with a small percentage of multivariate nodes close to the root generalize better than pure trees with the same type of node everywhere. CV produces simpler trees than AIC and BIC without sacrificing from expected error. The only disadvantage of CV is its longer training time. 1
Parallel univariate decision trees
, 2007
"... Univariate decision tree algorithms are widely used in data mining because (i) they are easy to learn (ii) when trained they can be expressed in rule based manner. In several applications mainly including data mining, the dataset to be learned is very large. In those cases it is highly desirable to ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Univariate decision tree algorithms are widely used in data mining because (i) they are easy to learn (ii) when trained they can be expressed in rule based manner. In several applications mainly including data mining, the dataset to be learned is very large. In those cases it is highly desirable to construct univariate decision trees in reasonable time. This may be accomplished by parallelizing univariate decision tree algorithms. In this paper, we first present two different univariate decision tree algorithms C4.5 and univariate linear discriminant tree. We show how to parallelize these algorithms in three ways: (i) feature based; (ii) node based; (iii) data based manners. Experimental results show that performance of the parallelizations highly depend on the dataset and the node based parallelization demonstrate good speedups.
Budding Trees
"... Abstract—We propose a new decision tree model, named the budding tree, where a node can be both a leaf and an internal decision node. Each bud node starts as a leaf node, can then grow children, but then later on, if necessary, its children can be pruned. This contrasts with traditional tree constru ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract—We propose a new decision tree model, named the budding tree, where a node can be both a leaf and an internal decision node. Each bud node starts as a leaf node, can then grow children, but then later on, if necessary, its children can be pruned. This contrasts with traditional tree construction algorithms that only grows the tree during the training phase, and prunes it in a separate pruning phase. We use a soft tree architecture and show that the tree and its parameters can be trained using gradientdescent. Our experimental results on regression, binary classification, and multiclass classification data sets indicate that our newly proposed model has better performance than traditional trees in terms of accuracy while inducing trees of comparable size. I.