Abstract

A ray-tracing method inspired by ergodic billiards is used to estimate the theoretically best decision rule for a given set of linear separable examples. For randomly distributed examples the billiard estimate of the single Perceptron with best average generalization probability agrees with known analytic results, while for real-life classification problems the generalization probability is consistently enhanced when compared to the maximal stability Perceptron. 1 Introduction Neural networks can be used for both concept learning (classification) and for function interpolation and/or extrapolation. Two basic mathematical methods seem to be particularly adequate for studying neural networks: geometry (especially combinatorial geometry) and probability theory (statistical physics). Geometry is illuminating and probability theory is powerful. In this paper I consider the perhaps simplest neural network, the venerable Perceptron [1]: given a set of examples falling in two classes,...

Citations