This work gives a polynomial time algorithm for learning decision trees with respect to the uniform distribution. (This algorithm uses membership queries.) The decision tree model that is considered is an extension of the traditional boolean decision tree model that allows linear operations in each node (i.e., summation of a subset of the input variables over GF (2)). This paper shows how to learn in polynomial time any function that can be approximated (in norm L 2 ) by a polynomially sparse function (i.e., a function with only polynomially many non-zero Fourier coefficients). The authors demonstrate that any function f whose L 1 -norm (i.e., the sum of absolute value of the Fourier coefficients) is polynomial can be approximated by a polynomially sparse function, and prove that boolean decision trees with linear operations are a subset of this class of functions. Moreover, it is shown that the functions with polynomial L 1 -norm can be learned deterministically. The algorithm can a...

Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial neural networks. Researchers in these disciplines, sometimes working on quite different problems, identified similar issues and heuristics for decision tree construction. This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art. Keywords: classification, tree-structured classifiers, data compaction 1. Introduction Advances in data collection methods, storage and processing technology are providing a unique challenge and opportunity for automated data exploration techniques. Enormous amounts of data are being collected daily from major scientific projects e.g., Human Genome...

We show that a DNF with terms of size at most d can be approximated by a function with at most d O(d log 1=") non zero Fourier coefficients such that the expected error squared, with respect to the uniform distribution, is at most ". This property is used to derive a learning algorithm for DNF, under the uniform distribution. The learning algorithm uses queries and learns, with respect to the uniform distribution, a DNF with terms of size at most d in time polynomial in n and d O(d log 1=") . The interesting implications are for the case when " is constant. In this case our algorithm learns a DNF with a polynomial number of terms in time n O(log log n) , and a DNF with terms of size at most O(log n= log log n) in polynomial time.

In this talk we will consider various classes defined by small depth polynomial size circuits which contain threshold gates and parity gates. We will describe various inclusions between many classes defined in this way and also classes whose definitions rely upon spectral properties of Boolean functions.

We extend the area of applications of the Abstract Harmonic Analysis to lower bounds on the circuit and decision tree complexity of Boolean functions related to some number theoretic problems. In particular, we prove that deciding if a given integer is square-free and testing co-primality of two integers by unbounded fan-in circuits of bounded depth requires superpolynomial size. 1 Introduction In recent years spectral techniques based on the Abstract Harmonic Analysis on the hypercube have been shown to represent a very useful tool for obtaining lower complexity bounds. Various links between Fourier coefficients of Boolean functions and their complexity characteristics have been studied in a number of works, see [1, 2, 3, 4, 6, 8, 13, 19, 20, 22, 23]. In particular, these spectral techniques have been successfully applied to the parity function and to threshold functions. Institut fur Informatik, Technische Universitat Munchen, D-80290 Munchen, Germany. bernasco@informatik.tu-mue...

A Naive (or Idiot) Bayes network is a network with a single hypothesis node and several observations that are conditionally independent given the hypothesis. We recently surveyed a number of members of the UAI community and discovered a general lack of understanding of the implications of the Naive Bayes assumption on the kinds of problems that can be solved by these networks. It has long been recognized [Minsky 61] that if observations are binary, the decision surfaces in these networks are hyperplanes. We extend this result (hyperplane separability) to Naive Bayes networks with m-ary observations. In addition, we illustrate the effect of observation-observation dependencies on decision surfaces. Finally, we discuss the implications of these results to knowledge acquisition and research in learning. 1 Introduction While it is widely accepted that Naive Bayes models are a weaker representation for classification problems than a more general belief network, the precise nature of the li...

Abstract We study combinatorial complexity characteristics of a Boolean function related to a natural number theoretic problem. In particular we obtain a linear lower bound on the average sensitivity of the Boolean function deciding whether a given integer is square-free. This result allows us to derive a quadratic lower bound for the formula size complexity of testing square-free numbers and a linear lower bound on the average decision tree depth. We also obtain lower bounds on the degrees of exact and approximative polynomial representations of this function. \Lambda Supported by DFG grant Me 1077/14-1.

We study the Fourier representation of Boolean functions. The goal is to look at the frequency domain of Boolean functions to get complexity properties. Preliminary results indicate that this might be fruitful. In addition to presenting new results, we review some of the most significant work on the subject. Istituto di Matematica Computazionale, Consiglio Nazionale delle Ricerche, and Dipartimento di Informatica, Pisa (Italy). y Istituto di Matematica Computazionale, Consiglio Nazionale delle Ricerche, Pisa (Italy). e-mail: codenotti@iei.pi.cnr.it z Department of Computer Science, The University of Chicago. Portions of this work were done while visiting IEI-CNR in Pisa, sponsored by a grant from CNR. 1 Introduction The Fourier transform of a Boolean function is an invertible linear mapping of the values of the function onto a set of coefficients, known as Fourier coefficients. This transformation is such that the Fourier coefficients contain information about the regularitie...

Abstract We study combinatorial complexity characteristics of a Boolean function related to a natural number theoretic problem. In particular we obtain an asymtotic formula, having a linear main term, for the average sensitivity of the Boolean function deciding whether a given integer is square-free. This result allows us to derive a quadratic lower bound for the formula size complexity of testing square-free numbers and a linear lower bound on the average decision tree depth. We also obtain lower bounds on the degrees of exact and approximative polynomial representations of this function. *Supported by DFG grant Me 1077/14-1.#

- Implications of our results and technique are discussed, for estimating the spectral norms of any function in a constant depth circuit class, using the coding theoretic concept of weight distributions. Evaluating the spectral norms for any such function reduces to estimating certain non-trivial weight distributions of simple, linear codes.