Citation Context

...es have an inherent diversity, would there be any benefit in using the measure of ambiguity as an optimization criteria in order to maximize performance? Can we significantly reduce the number of kNN in the ensemble, and thus reduce the cardinality of the EoC without any loss of performance in generalization? 2. Optimization methods In this work, we analyzed the effect of using a measure of diversity called ambiguity [4] in the optimization of ensembles of kNN generated by the random subspace method. The optimization is done using a multi-objective genetic algorithm and the NIST SD19 database [1]. The data consists of handwritten numerals forming a 10 classes pattern recognition problem. Prior to the optimization, we have experimentally determined that a one-nearest neighbour classifier (k=1) in a subspace of 32 features (out of 132) was the best combination of parameters for our ensembles of kNN. Performance on the Optimization set 86% 88% 90% 92% 94% 96% 98% 0 10000 20000 30000 40000 50000 number of prototypes R ec og ni ti on r at e Ensemble of 100 kNN (k=1) Single kNN (k=1) MLP Figure 1: Accuracy evolution as the training set increases for MLP, kNN and ensemble of 100 kNN in rando...

Citation Context

...ing and sensitivity towards the weights. It has been demonstrated that feature selection through multi-objective genetic algorithm (MOGA) is a very powerful tool for finding a set of good classifiers =-=[4,14]-=-, since GA is quite effective in rapid global search of large, non-linear and poorly understood spaces [17]. Besides, it can overcome problems such as scaling and sensitivity towards the weights. Kudo...

Citation Context

...rategy is able to select P1 in Fig. 2 as the most accurate generalization solution. It produces better results than selecting solutions based solely on the accuracy of the optimization data set alone =-=[6]-=-–[8]. In spite of this, the use of a selection data set can not prevent dominated solutions D1 and D2 which has great generalization power from being discarded by the MOGA. The main shortcoming of thi...

Citation Context

...ion. Genetic Algorithms (GAs), used for feature selection, and based on the wrapper method [4], need a classifier (SVM, Neural network, Near-Neighbour..) to evaluate each individual of the population =-=[1, 6, 11]-=-. But training classifiers at each iteration of GA is too much time consuming. In this paper, we propose a new feature selection method, based on GA, which avoid classifier training at each iteration ...

Citation Context

...sumption about the properties ofsthe evaluation function. Multiobjective evaluation functionss(e.g. combining the accuracy and the cost of classification) cansbe defined and used in a natural way [6] =-=[7]-=-. GAs can easilysencode decisions about selecting or not selecting features asssequences of boolean values,sallow to smartly explore thesfeature space by retaining those decisions that benefit thescla...

Citation Context

... the master-slave type without prior modification. The highest efficiency of the master-slave parallelization type can be achieved on computers with homogeneous processors and in problem domains where the fitness evaluation time is long, constant, and independent of the solution. When these criteria are fulfilled, near-linear speedup (Akl, 1997) (speedup that is close to the upper 4 Evolutionary Computation Volume x, Number x AMS Parallelization of DEMO theoretical limit) is possible. The master-slave parallelization is popular with MOEAs, ranging from simple implementations as in the case of Oliveira et al. (2003) where the master runs on a separate processor, and the cases of Radtke et al. (2003) and Nebro and Durillo (2010) where the master node also runs one slave. There are also implementations for heterogeneous computer architectures where load-balancing has to be implemented. Examples can be found in Eberhard et al. (2003) with pool-of-tasks load balancing algorithm, Lim et al. (2007) where a grid-enabled algorithm combines the island model with the master-slave model, Stanley and Mudge (1995), and Talbi and Meunier (2006) with an asynchronous master-slave parallelization of a steady-state algori...

Citation Context

...attractive approach to feature selection since they can generally perform quite an effective search of a large, non-linear space [104]. In the handwritten character recognition area, some researchers =-=[51, 86]-=- have developed OCR-oriented criteria or fitness functions, which can alleviate the computation complexity for a given feature number m, (m≤n, n is the number of features initially extracted). However...