There is an ongoing debate over the capabilities of hierarchical neural feed-forward architectures for performing real-world invariant object recognition. Although a variety of hierarchical models exists, appropriate supervised and unsupervised learning methods are still an issue of intense research. We propose a feedforward model for recognition that shares components like weightsharing, pooling stages, and competitive nonlinearities with earlier approaches, but focus on new methods for learning optimal featuredetecting cells in intermediate stages of the hierarchical network.

We propose a cortically inspired hierarchical feedforward model for recognition and investigate a new method for learning optimal combination-coding cells in intermediate stages of the hierarchical network.

We consider recurrent neural networks which deal with symbolic formulas, terms, or, generally speaking, tree-structured data. Approaches like the recursive autoassociative memory, discrete-time recurrent networks, folding networks, tensor construction, holographic reduced representations, and recursive reduced descriptions fall into this category. They share the basic dynamics of how structured data are processed: the approaches recursively encode symbolic data into a connectionistic representation or decode symbolic data from a connectionistic representation by means of a simple neural function. In this paper, we give an overview of the ability of neural networks with these dynamics to encode and decode tree-structured symbolic data. The correlated tasks, approximating and learning mappings where the input domain or the output domain may consist of structured symbolic data, are examined as well.

We suggest a new approach to optimize the learning of sparse features under the constraints of explicit transformation symmetries imposed on the set of feature vectors. Given a set of basis feature vectors and invariance transformations, from each basis feature a family of transformed features is generated. We then optimize the basis features for optimal sparse reconstruction of the input pattern ensemble using the whole transformed feature family. If the predened transformation invariance coincides with an invariance in the input data, we obtain a less redundant basis feature set, compared to sparse coding approaches without invariances. We demonstrate the application to a test scenario of overlapping bars and the learning of receptive elds in hierarchical visual cortex models.

We present a new approach to the supervised learning of lateral interactions for the competitive layer model (CLM) dynamic feature binding architecture. The method is based on consistency conditions, which were recently shown to characterize the attractor states of this linear threshold recurrent network. For a given set of training examples the learning problem is formulated as a convex quadratic optimization problem in the lateral interaction weights. An efficient dimension reduction of the learning problem can be achieved by using a linear superposition of basis interactions.

Abstract — We present a hybrid learning method bridging the fields of recurrent neural networks, unsupervised Hebbian learning, vector quantization, and supervised learning to implement a sophisticated image and feature segmentation architecture. This architecture is based on the competitive layer model (CLM) dynamic feature binding model which is applicable on a wide range of perceptual grouping and segmentation problems. A predefined target segmentation can be achieved as attractor states of this linear threshold recurrent network, if the lateral weights are chosen by Hebbian learning. The weight matrix is given by the correlation matrix of special pattern vectors with a structure dependent on the target labeling. Generalization is achieved by applying vector quantization on pairwise feature relations, like proximity and similarity, defined by external knowledge. We show the successful application of the method to a number of artifical test examples and a medical image segmentation problem of fluorescence microscope cell images.

We address the problem of fast figure-ground segmentation of single objects from cluttered backgrounds to improve object learning and recognition. For the segmentation, we use an initial foreground hypothesis to train a classifier for figure and ground on topographically ordered feature maps with Generalized Learning Vector Quantization. We investigate the contribution of several adaptive metrics to enable generalization to the main object parts and derive a foreground classification, which yields an improved bottom-up hypothesis. We show that metrics adaptation is a powerful enrichment, where generalizing the Euclidean metrics towards local matrices of relevance-factors leads to a higher classification accuracy and considerable robustness on partially inconsistent supervised information. Additionally, we verify our results in an online learning scenario and show that figure-ground segregation using this adaptive metrics enables a considerably higher recognition performance on segmented object views. Key words: relevance learning, figure-ground segregation, generalized learning vector quantization, object recognition 1

A mathematical analysis of a special class of winner-take-all networks is given. Starting with a solution in closed form describing the dynamics of the network, we show that an inhibitory winner-take-all net efficiently discriminates input patterns with respect to a canonical measure. This result in combination with further properties suggests an upgrading of the system by incorporating fault tolerance and efficiently generated evidential response into the system.

We present a novel self-organizing classification network for structured objects. The system consists of two layers, an upper layer which serves as a classifier and a lower layer as a feature extractor. The classifier is an inhibitory winner-take-all network where each unit corresponds to a unique class while the lower layer consists of Hopfield-style networks solving the problem of finding maximum common parts of the input structure and the particular prototypes. The self-organizing architecture is able to outperform traditional classification models using a maximum selector as a classifier. The improvements arise from deactivating networks in the lower layer which indicate large dissimilarities between the input structure and the corresponding prototypes.

We present an overview of current lines of research on learning with recurrent neural networks (RNNs). Topics covered are: understanding and unification of algorithms, theoretical foundations, new efforts to circumvent gradient vanishing, new architectures, and fusion with other learning methods and dynamical systems theory. The structuring guideline is to understand many new approaches as different efforts to regularize and thereby improve recurrent learning.