Results 1  10
of
32
Neural Associative Memories
 Biological Cybernetics
, 1993
"... Despite of processing elements which are thousands of times faster than the neurons in the brain, modern computers still cannot match quite a few processing capabilities of the brain, many of which we even consider trivial (such as recognizing faces or voices, or following a conversation). A common ..."
Abstract

Cited by 110 (14 self)
 Add to MetaCart
Despite of processing elements which are thousands of times faster than the neurons in the brain, modern computers still cannot match quite a few processing capabilities of the brain, many of which we even consider trivial (such as recognizing faces or voices, or following a conversation). A common principle for those capabilities lies in the use of correlations between patterns in order to identify patterns which are similar. Looking at the brain as an information processing mechanism with  maybe among others  associative processing capabilities together with the converse view of associative memories as certain types of artificial neural networks initiated a number of interesting results, ranging from theoretical considerations to insights in the functioning of neurons, as well as parallel hardware implementations of neural associative memories. This paper discusses three main aspects of neural associative memories: ffl theoretical investigations, e.g. on the information storage...
Cell Assemblies, Associative Memory and Temporal Structure in Brain Signals
"... : In this work we discuss Hebb's old ideas about cell assemblies in the light of recent results concerning temporal structure and correlations in neural signals. We want to give a conceptual, necessarily only rough picture, how ideas about `binding by synchronisation', `synfire chains&apos ..."
Abstract

Cited by 25 (7 self)
 Add to MetaCart
: In this work we discuss Hebb's old ideas about cell assemblies in the light of recent results concerning temporal structure and correlations in neural signals. We want to give a conceptual, necessarily only rough picture, how ideas about `binding by synchronisation', `synfire chains', `local and global assemblies', `short and long term memory' and `behaviour' might be integrated into a coherent model of brain functioning based on neuronal assemblies. Keywords: cell assemblies, synchronization, gammaoscillations, synfire chains, memory, behaviour 1 ASSEMBLIES AND ASSOCIATIVE MEMORIES 1.1 Cell Assemblies Cell assemblies have been introduced by Donald Hebb with the intention of providing a functional and at the same time structural model for cortical processes and neuronal representations of external events (Hebb, 1949). According to Hebb's ideas, stimuli, objects, things, but also more abstract entities like concepts, contextual relations, ideas, and so on are thought of being repre...
Scene Segmentation by Spike Synchronization in Reciprocally Connected Visual Areas I. Local Effects of Cortical Feedback
 Biological Cybernetics
, 2002
"... To investigate scene segmentation in the visual system we present a model of two reciprocally connected visual areas using spiking neurons. Area P corresponds to the orientation selective subsystem of the primary visual cortex, while the central visual area C is modeled as associative memory represe ..."
Abstract

Cited by 24 (5 self)
 Add to MetaCart
To investigate scene segmentation in the visual system we present a model of two reciprocally connected visual areas using spiking neurons. Area P corresponds to the orientation selective subsystem of the primary visual cortex, while the central visual area C is modeled as associative memory representing stimulus objects according to Hebbian learning. Without feedback from area C, a single stimulus results in relatively slow and irregular activity, synchronized only for neighboring patches (slow state), while in the complete model activity is faster with enlarged synchronization range (fast state). Presenting a superposition of several stimulus objects, scene segmentation happens on a time scale of hundreds of milliseconds by alternating epochs of the slow and fast state, where neurons representing the same object are simultaneously in the fast state. Correlation analysis reveals synchronization on different time scales as found in experiments (T,C,H peaks). On the fast time scale (T peaks, gamma frequency range), recordings from two sites coding either different or the same object lead to correlograms that are either at or exhibit oscillatory modulations with a central peak. This is in agreement with experimental findings while standard phase coding models would predict shifted peaks in the case of different objects.
Matching Performance of Binary Correlation Matrix Memories
"... We introduce a theoretical framework for estimating the matching performance of binary correlation matrices acting as heteroassociative memories. The framework is applicable to nonrecursive, fullyconnected systems with binary (0,1) Hebbian weights and hardlimited threshold. It can handle both fu ..."
Abstract

Cited by 20 (12 self)
 Add to MetaCart
We introduce a theoretical framework for estimating the matching performance of binary correlation matrices acting as heteroassociative memories. The framework is applicable to nonrecursive, fullyconnected systems with binary (0,1) Hebbian weights and hardlimited threshold. It can handle both full and partial matching of single or multiple data items in nonsquare memories. Theoretical development takes place under a probability theory framework. Inherent uncertainties in the matching process are accommodated by the use of probability distributions to describe the numbers of correct and incorrect neuron responses during retrieval. Theoretical predictions are verified experimentally for mediumsized memories and used to aid the design of larger systems. The results highlight the Matching Performance of CMMs 2 fact that correlationbased models can act as highly efficient memories provided a small probability of retrieval error is accepted. Keywords Neural Associative Memories, Co...
Associative Memory in Networks of Spiking Neurons
, 2001
"... Here we develop and investigate a computational model of a network of cortical neurons on the base of biophysically well constrained and tested twocompartmental neurons developed by Pinsky and Rinzel [Pinsky and Rinzel, 1994]. To study associative memory we connect a pool of cells by a structure ..."
Abstract

Cited by 19 (2 self)
 Add to MetaCart
Here we develop and investigate a computational model of a network of cortical neurons on the base of biophysically well constrained and tested twocompartmental neurons developed by Pinsky and Rinzel [Pinsky and Rinzel, 1994]. To study associative memory we connect a pool of cells by a structured connectivity matrix. The connection weights are shaped by simple Hebbian coincidence learning using a set of spatially sparse patterns. We study the neuronal activity processes following an external stimulation of a stored memory. In two series of simulation experiments we explore the effect of different classes of external input, tonic and flashed stimulation: With tonic stimulation the addressed memory is attractor of the network dynamics. The memory is displayed rhythmically, coded by phase locked bursts or regular spikes. The participating neurons have rhythmic activity in the gammafrequency range (3080 Hz). If the input is switched from one memory to another, the network act...
On the relation between neural modelling and experimental neuroscience
, 1997
"... This paper discusses the relation of theory and experiment in neuroscience exemplified by three assumptions often made in models of coherent activation in the cortex: basic featurecoding oscillators, phasecoding and global binding of whole objects. Apparently these assumptions are not very well s ..."
Abstract

Cited by 17 (14 self)
 Add to MetaCart
(Show Context)
This paper discusses the relation of theory and experiment in neuroscience exemplified by three assumptions often made in models of coherent activation in the cortex: basic featurecoding oscillators, phasecoding and global binding of whole objects. Apparently these assumptions are not very well supported by the experimental evidence. We propose that it is the single synchronized populationburst that matters: spikes of featurecoding cells are temporally clustered in our opinion by recurrent associative processes. In each burst a single stimulus is processed (if there are several). Synchronization is restricted to cortical sites which physically interact. These principles are illustrated by computer simulations.
Bayesian Retrieval in Associative Memories with Storage Errors
 IEEE Trans. Neural Networks
, 1998
"... It is well known that for finitesized networks, onestep retrieval in the autoassociative Willshaw net is a suboptimal way to extract the information stored in the synapses. Iterative retrieval strategies are much better, but have hitherto only had heuristic justification. We show how they emerge ..."
Abstract

Cited by 15 (8 self)
 Add to MetaCart
(Show Context)
It is well known that for finitesized networks, onestep retrieval in the autoassociative Willshaw net is a suboptimal way to extract the information stored in the synapses. Iterative retrieval strategies are much better, but have hitherto only had heuristic justification. We show how they emerge naturally from considerations of probabilistic inference under conditions of noisy and partial input and a corrupted weight matrix. We start from the conditional probability distribution over possible patterns for retrieval. This contains all possible information that is available to an observer of the network and the initial input. Since this distribution is over exponentially many patterns, we use it to develop two approximate, but tractable, iterative retrieval methods. One performs maximum likelihood inference to find the single most likely pattern, using the (negative log of the) conditional probability as a Lyapunov function for retrieval. In physics terms, if storage errors are present, then the modified iterative update equations contain an additional antiferromagnetic interaction term and site dependent threshold values. The second method makes a mean field assumption to optimize a tractable estimate of the full conditional probability distribution. This leads to iterative mean field equations which can be interpreted in terms of a network of neurons with sigmoidal responses but with the same interactions and thresholds as in the maximum likelihood update equations. In the absence of storage errors, both models become very similiar to the Willshaw model, where standard retrieval is iterated using a particular form of linear threshold strategy.
Gammaoscillations support optimal retrieval in associative memories of PinskyRinzel neurons
, 1998
"... Theoretical studies concerning iterative retrieval in conventional associative memories suggest that cortical gammaoscillations may constitute sequences of fast associative processes each restricted to a single period. By providing a rhythmic threshold suppressing cells uncorrelated with a stim ..."
Abstract

Cited by 7 (6 self)
 Add to MetaCart
Theoretical studies concerning iterative retrieval in conventional associative memories suggest that cortical gammaoscillations may constitute sequences of fast associative processes each restricted to a single period. By providing a rhythmic threshold suppressing cells uncorrelated with a stimulus, interneurons significiantly contribute to this process. This hypothesis is tested in the present paper utilizing a network of reduced compartment neurons developed by Pinsky and Rinzel. It is shown that gammaoscillations can simultaneously support an optimal operation speed for pattern retrieval and a very high memory capacity. Keywords: gammaoscillations, threshold control, interneurons, associative memory, compartmental modeling 1 Introduction Synchronized gammaoscillations have been observed in different species and many cortical areas [1,7]. In sensory areas they have been interpreted by the Submitted to Computational Neuroscience Meeting, CNS98, Santa Barbara, July 2530...
Efficient Covariance Matrix Methods for Bayesian Gaussian Processes and Hopfield Neural Networks
, 1999
"... Covariance matrices are important in many areas of neural modelling. In Hopfield networks they are used to form the weight matrix which controls the autoassociative properties of the network. In Gaussian processes, which have been shown to be the infinite neuron limit of many regularised feedforward ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
Covariance matrices are important in many areas of neural modelling. In Hopfield networks they are used to form the weight matrix which controls the autoassociative properties of the network. In Gaussian processes, which have been shown to be the infinite neuron limit of many regularised feedforward neural networks, covariance matrices control the form of Bayesian prior distribution over function space. This thesis examines interesting modifications to the standard covariance matrix methods to increase functionality or efficiency of these neural techniques. Firstly the problem of adapting Gaussian process priors to perform regression on switching regimes is tackled. This involves the use of block covariance matrices and Gibbs sampling methods. Then the use of Toeplitz methods is proposed for Gaussian process regression where sampling positions can be chosen. A comparison is made between Hopfield weight matrices, and sample covariances. This allows work on sample covariances to be used ...
Models of distributed associative memory networks in the brain
 THEORY IN BIOSCIENCES
, 2003
"... Although experimental evidence for distributed cell assemblies is growing, theories of cell assemblies are still marginalized in theoretical neuroscience. We argue that this has to do with shortcomings of the currently best understood assembly theories, the ones based on formal associative memory mo ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
Although experimental evidence for distributed cell assemblies is growing, theories of cell assemblies are still marginalized in theoretical neuroscience. We argue that this has to do with shortcomings of the currently best understood assembly theories, the ones based on formal associative memory models. These only insufficiently reflect anatomical and physiological properties of nervous tissue and their functionality is too restricted to provide a framework for cognitive modeling. We describe cell assembly models that integrate more neurobiological constraints and review results from simulations of a simple nonlocal associative network formed by a reciprocal topographic projection. Impacts of nonlocal associative projections in the brain are discussed with