The problem of statistical learning can be considered as a problem of nonparametric estimation of sets, where the risk is de ned by means of a speci c distance function between sets associated to the misclassi cation error. The rates of convergence of classi ers depend on two parameters: the complexity of the class of candidate sets and the "margin" parameter. The dependence is explicitly given, in particular the optimal rates up to O(n ) can be attained, where n is the sample size, and the proposed classi ers have the property of robustness to the margin. The main result of the paper concerns optimal aggregation of classi ers: we suggest a classi er that automatically adapts both to the complexity and to the margin, and attains the optimal fast rates, up to a logarithmic factor.

Artificial Neural Networks (ANNs) are very popular as classification or regression mechanisms in medical decision support systems despite the fact that they are unstable predictors. This instability means that small changes in the training data used to build the model (i.e. train the ANN) may result in very different models. A central implication of this is that different sets of training data may produce models with very different generalisation accuracies. In this paper we show in detail how this can happen in a prediction system for use in In-Vitro Fertilisation. We argue that claims for the generalisation performance of ANNs used in such a scenario should only be based on k-fold cross validation tests. We also show how the accuracy of such a predictor can be improved by aggregating the output of several predictors. 1. Introduction Artificial Neural Networks (ANNs) are hugely popular in research on medical decision support systems (see Baxt's review of clinical applicatio...

It is argued that for the computer to be able to interact with humans, it needs to have the communication skills of humans. One of these skills is the ability to understand the emotional state of the person. The most expressive way humans display emotions is through facial expressions. In most facial expression systems and databases, the emotion data was collected by asking the subjects to perform a series of facial expressions. However, these directed or deliberate facial action tasks typically differ in appearance and timing from the authentic facial expressions induced through events in the normal environment of the subject. In this paper, we present our effort in creating an authentic facial expression database based on spontaneous emotions derived from the environment. Furthermore, we test and compare a wide range of classifiers from the machine learning literature that can be used for facial expression classification.

A framework for Similarity-Based Methods (SBMs) includes many classification models as special cases: neural network of the Radial Basis Function Networks type, Feature Space Mapping neurofuzzy networks based on separable transfer functions, Learning Vector Quantization, variants of the k nearest neighbor methods and several new models that may be presented in a network form. Multilayer Perceptrons (MLPs) use scalar products to compute weighted activation of neurons, combining soft hyperplanes to provide decision borders. Distance-based multilayer perceptrons (D-MLPs) evaluate similarity of inputs to weights offering a natural generalization of standard MLPs. Cluster-based initialization procedure determining architecture and values of all adaptive parameters is described. Networks

Numerous empirical results have shown that combining regression procedures can be a very ecient method. This work provides PAC bounds for the L generalization error of such methods. The interest of these bounds are twofold. First, it gives for any aggregating procedure a bound for the expected risk depending on the empirical risk and the empirical complexity measured by the Kullback-Leibler divergence between the aggregating distribution ^ and a prior distribution and by the empirical mean of the variance of the regression functions under the probability ^ .

In this paper we address the issue of how to optimize the generalization performance of bagged neural network ensembles. We investigate how diversity amongst networks in bagged ensembles can significantly influence ensemble generalization performance and propose a new early-stopping technique that effectively tunes this diversity so that overall ensemble generalization performance is optimized. Experiments performed on benchmark regression data-sets demonstrate the potential of the technique. Keywords: Bagging, diversity, ensemble, generalization, early-stopping 1 Introduction Recently, neural network ensemble techniques have gained widespread use amongst neural network practitioners. There are many different varieties, but the most popular include some elaboration of bagging [2], boosting [11] or stacking [34]. The basic idea of these techniques is to generate multiple versions of a predictor. When predictions from these versions are combined (averaged for example), smoother more ...

. The naive Bayesian classifier provides a very simple yet surprisingly accurate technique for machine learning. Some researchers have examined extensions to the naive Bayesian classifier that seek to further improve the accuracy. For example, a naive Bayesian tree approach generates a decision tree with one naive Bayesian classifier at each leaf. Another example is a constructive Bayesian classifier that eliminates attributes and constructs new attributes using Cartesian products of existing attributes. This paper proposes a simple, but effective approach for the same purpose. It generates a naive Bayesian classifier committee for a given classification task. Each member of the committee is a naive Bayesian classifier based on a subset of all the attributes available for the task. During the classification stage, the committee members vote to predict classes. Experiments across a wide variety of natural domains show that this method significantly increases the prediction accuracy of t...

Neural network ensemble can significantly improve the generalization ability of neural network based systems. However, its comprehensibility is even worse than that of a single neural network because it comprises a collection of individual neural networks. In this paper, an approach named REFNE is proposed to improve the comprehensibility of trained neural network ensembles that perform classification tasks. REFNE utilizes the trained ensembles to generate instances and then extracts symbolic rules from those instances. It gracefully breaks the ties made by individual neural networks in prediction. It also employs specific discretization scheme, rule form, and fidelity evaluation mechanism. Experiments show that with different configurations, REFNE can extract rules with good fidelity that well explain the function of trained neural network ensembles, or rules with strong generalization ability that are even better than the trained neural network ensembles in prediction.

Classification systems play an important role in business decision-making tasks by classifying the available information based on some criteria. The objective of this research is to assess the relative performance of some well-known classification methods. We consider classification techniques that are based on statistical and AI techniques. We use synthetic data to perform a controlled experiment in which the data characteristics are systematically altered to introduce imperfections such as nonlinearity, multicollinearity, unequal covariance, etc. Our experiments suggest that data characteristics considerably impact the classification performance of the methods. The results of the study can aid in the design of classification systems in which several classification methods can be employed to increase the reliability and consistency of the classification.

Abstract. One advantage of evolutionary multiobjective optimization (EMO) algorithms over classical approaches is that many non-dominated solutions can be simultaneously obtained by their single run. In this paper, we propose an idea of using EMO algorithms for constructing an ensemble of fuzzy rule-based classifiers with high diversity. The classification of new patterns is performed based on the vote of multiple classifiers generated by a single run of EMO algorithms. Even when the classification performance of individual classifiers is not high, their ensemble often works well. The point is to generate multiple classifiers with high diversity. We demonstrate the ability of EMO algorithms to generate various non-dominated fuzzy rule-based classifiers with high diversity by their single run. Through computational experiments on some wellknown benchmark data sets, it is shown that the vote of generated fuzzy rulebased classifiers leads to high classification performance on test patterns. 1