This paper provides an introduction to support vector machines (SVMs), kernel Fisher discriminant analysis, and

Abstract — In this paper, we propose the use of Support Vector Machines (SVM) for automatic hyperspectral data classification and knowledge discovery. In the first stage of the study, we use SVMs for crop classification and analyze their performance in terms of efficiency and robustness, as compared to extensively used neural and fuzzy methods. Efficiency is assessed by evaluating accuracy and statistical differences in several scenes. Robustness is analyzed in terms of (a) suitability to working conditions when a feature selection stage is not possible, and (b) performance when different levels of Gaussian noise are introduced at their inputs. In the second stage of this work, we analyze the distribution of the support vectors (SV) and perform sensitivity analysis on the best classifier in order to analyze the significance of the input spectral bands. For classification purposes, six hyperspectral images acquired with the 128-band HyMAP spectrometer during the DAISEX-1999 campaign are used. Six crop classes were labelled for each image. A reduced set of labelled samples is used to train the models and the entire images are used to assess their performance. Several conclusions are drawn: (1) SVMs yield better outcomes than neural networks regarding accuracy, simplicity and robustness; (2) training neural and neurofuzzy models is unfeasible when working with high dimensional input spaces and great amounts of training data; (3) SVMs perform similarly for different training subsets with varying input dimension, which indicates that noisy bands are successfully detected; and (4) a valuable ranking of bands through sensitivity analysis is achieved. Index Terms — Hyperspectral imagery, crop classification, knowledge discovery, Support Vector Machines, neural networks.

Exemplar theories of categorization depend on similarity for explaining subjects’ ability to generalize to new stimuli. A major criticism of exemplar theories concerns their lack of abstraction mechanisms and thus, seemingly, of generalization ability. Here, we use insights from machine learning to demonstrate that exemplar models can actually generalize very well. Kernel methods in machine learning are akin to exemplar models and are very successful in real-world applications. Their generalization performance depends crucially on the chosen similarity measure. Although similarity plays an important role in describing generalization behavior, it is not the only factor that controls generalization performance. In machine learning, kernel methods are often combined with regularization techniques in order to ensure good generalization. These same techniques are easily incorporated in exemplar models. We show that the generalized context model (Nosofsky, 1986) and ALCOVE (Kruschke, 1992) are closely related to a statistical model called kernel logistic regression. We argue that generalization is central to the enterprise of understanding categorization behavior, and we suggest some ways in which insights from machine learning can offer guidance.

Shortcut connections are a popular architectural feature of multi-layer perceptrons. It is generally assumed that by implementing a linear submapping, shortcuts assist the learning process in the remainder of the network. Here we find that this is not always the case: shortcut weights may also act as distractors that slow down convergence and can lead to inferior solutions. This problem can be addressed with slope centering, a particular form of gradient factor centering [2]. By removing the linear component of the error signal at a hidden node, slope centering effectively decouples that node from the shortcuts that bypass it. This eliminates the possibility of destructive interference from shortcut weights, and thus ensures that the benefits of shortcut connections are fully realized.

Permission is hereby granted to the University of Alberta Library to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. The author reserves all other publication and other rights in association with the copyright in the thesis, and except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatever without the author’s prior written permission. Date:

This paper has four threads which tie together the business of delivering the findings of loyalty studies to commercial clients. The threads emerged from a loyalty survey for which traditional analysis yielded no significant findings. The model problem arose from a lack of agreement between common assumptions made in traditional analysis (eg, linear, quasi-linear), and the semantics of the behaviour/belief structure underlying loyalty. The findings are applicable to other psychometric models derived from surveys, including choice, preference and rank preference, and other forms of declared intent models. The threads: 1 The need for sophisticated non-linear models to ‘fit ’ complex customer and market behaviours. 2 The drawback of these advanced approaches is a loss of the ability to explain customer and market behaviours with simple ‘main effect ’ co-efficients. Business must follow science in recognising the dangers of trying to summarise complex phenomena through simplistic and highly restrictive quantitative methods. 3 The authors argue a case for robust, sophisticated methods in conjunction with model

Motivation: Human decisions often proceed in two steps. Initially those most preferred are chosen followed by a subsequent choice of these preferences. Applying one artificial neural network (ANN), a classification is limited to the preselection process. The final categorization is only possible by a subsequent ANN that distinguishes the pre-chosen classes. Existing strategies using coupled ANNs are discussed and a new approach particularly suited for multiclass classification problems is introduced (‘Subsequent ANN’, SANN). Results: Evaluating a simulated data base comprising 3 classes, classification results of SANN were obviously superior to those achieved by ANN. To evaluate a real-world data base the microarray benchmark GCM (14 classes) was chosen. The ANN results reached 72%, comparable to previous results. Using SANN, up to 81 % of the tumors were correctly classified. Availability: Programs used in this work and numerical results are available upon request. Contact:

Abstract:- This paper studies the recognition of Persian handwritten characters using templates and back propagation networks. The last one is learned by gradient decent learning law which was promoted by adaptive learning rate and momentum. Different technical methods which are often based on artificial neural network or neuro-fuzzy ones are used in recognition characters. Often the whole data is squeezed in the aforementioned networks which are to classify the data to each of existed classes. However, in this paper the templates are used for the primary classification of data. So, using templates leads to the recognition of some squeezed data and classification of remaining one into smaller common classes before feeding them into neural network; then the neural network is used for final classification. The results show that there are some significant improvements on recognition performance. This happens because decreasing input and output space causes to have a simpler mapping between input and output which in turn lead to increasing learning rate and decreasing classification error. Recognition rate on our dataset is almost 100%. Keywords:- Persian handwritten digits recognition; Templates; Back-propagation neural network; Adaptive learning rate; Momentum; Pattern recognition

www.elsevier.com/locate/physletb

Micro-robots, unmanned aerial vehicles (UAVs), imaging sensor networks, wireless phones, and other embedded vision systems all require low cost and high-speed implementations of synthetic vision systems capable of recognizing and categorizing objects in a scene. Many successful object recognition systems use dense features extracted on regularly-spaced patches over the input image. The majority of the feature extraction systems have a common structure composed of a filter bank (generally based on oriented edge detectors or 2D gabor functions), a non-linear operation (quantization, winner-take-all, sparsification, normalization, and/or point-wise saturation) and finally a pooling operation (max, average or histogramming). For example, the scale-invariant feature transform (SIFT (Lowe, 2004)) operator applies oriented edge filters to a small patch and determines the dominant orientation through a winner-take-all operation. Finally, the resulting sparse vectors are added (pooled) over a larger patch to form local orientation histogram. Some recognition systems use a single stage of feature extractors (Lazebnik