Abstract

We propose an Extreme Point Tabu Search (EPTS) algorithm that constructs globally optimal decision trees for classification problems. Typically, decision tree algorithms are greedy. They optimize the misclassification error of each decision sequentially. Our non-greedy approach minimizes the misclassification error of all the decisions in the tree concurrently. Using Global Tree Optimization (GTO), we can optimize existing decision trees. This capability can be used in classification and data mining applications to avoid overfitting, transfer knowledge, incorporate domain knowledge, and maintain existing decision trees. Our method works by fixing the structure of the decision tree and then representing it as a set of disjunctive linear inequalities. An optimization problem is constructed that minimizes the errors within the disjunctive linear inequalities. To reduce the misclassification error, a nonlinear error function is minimized over a polyhedral region. We show that it is suffici...

Citations