. We consider learning on multi-layer neural nets with piecewise polynomial activation functions and a fixed number k of numerical inputs. We exhibit arbitrarily large network architectures for which efficient and provably successful learning algorithms exist in the rather realistic refinement of Valiant's model for probably approximately correct learning ("PAC-learning") where no a-priori assumptions are required about the "target function" (agnostic learning), arbitrary noise is permitted in the training sample, and the target outputs as well as the network outputs may be arbitrary reals. The number of computation steps of the learning algorithm LEARN that we construct is bounded by a polynomial in the bit-length n of the fixed number of input variables, in the bound s for the allowed bit-length of weights, in 1 " , where " is some arbitrary given bound for the true error of the neural net after training, and in 1 ffi where ffi is some arbitrary given bound for the probability t...

This paper discusses within the framework of computational learning theory the current state of knowledge and some open problems in three areas of research about learning on feedforward neural nets: -- Neural nets that learn from mistakes -- Bounds for the Vapnik-Chervonenkis dimension of neural nets -- Agnostic PAC-learning of functions on neural nets. All relevant definitions are given in this paper, and no previous knowledge about computational learning theory or neural nets is required. We refer to [RSO] for further introductory material and survey papers about the complexity of learning on neural nets. Throughout this paper we consider the following rather general notion of a (feedforward) neural net.

This paper discusses within the framework of computational learning theory the current state of knowledge and some open problems in three areas of research about learning on feedforward neural nets:-- Neural nets that learn from mistakes-- Bounds for the Vapnik-Chervonenkis dimension of neural nets-- Agnostic PAC-learning of functions on neural nets. All relevant definitions are given in this paper, and no previous knowledge about computational learning theory or neural nets is required. We refer to [RSO] for further introductory material and survey papers about the complexity of learning on neural nets. Throughout this paper we consider the following rather general notion of a (feedforward) neural net. Definition 1.1 A network architecture (or "neural net") N is a labeled acyclic directed graph. Its nodes of fan-in 0 ( " input nodes"), as well as its nodes of fan-out 0 ( " output nodes") are labeled by natural numbers. A node g in N with fan-in r? 0 is called a computation node (or gate), and it is labeled by some activation function fl g: R! R, some polynomial Q g (y 1; : : : ; y r), and a subset P g of the coefficients of this polynomial (if P g is not separately specified we assume that P g consists of all coefficients of Q g). One says that N is of order v if all polynomials Q g in N are of degree v. The coefficients in the sets P g for the gates g in N are called the programmable parameters of N. Assume that N has w programmable parameters, that some numbering of these has been fixed, and that values for all non-programmable parameters have been assigned. Furthermore assume that N has d input nodes and l output nodes. Then each assignment ff 2 R w of reals to the programmable parameters in N defines an analog circuit N ff, which computes a function x 7! N ff

We put Bayes decision theory into the framework of pac-learning as introduced by Valiant [Val84]. Unlike classical Boolean concept learning where functions f : f0; 1g n ! f0; 1g are approximated, we assume here that f(¯x) is 0 (or 1) with a certain probability. We develop a theoretical framework for estimating functions and reduce the classification problem to the problem of estimating parameters. Within this framework it is shown that classifications based on n conditional independent Boolean features can efficiently be learned by examples. Our learning algorithm achieves with probability 1 \Gamma ffi an error which comes arbitrarily close (up to an additive ") to the optimal one of a perfect Bayes decision. It requires O i n 3 " 4 ln \Gamma n ffi \Delta j examples. In the particular case of two state classification, learning can be performed on a single neuron. Moreover we relax the restriction of conditional independence to dependencies of bounded order k and show that in...

Computing the maximum bichromatic discrepancy is an interesting theoretical problem with important applications in computational learning theory, computational geometry and computer graphics. In this paper we give algorithms to compute the maximum bichromatic discrepancy for simple geometric ranges, including rectangles and halfspaces. In addition, we give extensions to other discrepancy problems. 1 Introduction The main theme of this paper is to present efficient algorithms that solve the problem of computing the maximum bichromatic discrepancy for axis oriented rectangles. This problem arises naturally in different areas of computer science, such as computational learning theory, computational geometry and computer graphics ([Ma], [DG]), and has applications in all these areas. In computational learning theory, the problem of agnostic PAC-learning with simple geometric hypotheses can be reduced to the problem of computing the maximum bichromatic discrepancy for simple geometric ra...