Willshaw, D. J., Buneman, & Longuet-Higgins, H. C. (1969). Nonholographic associative memory. Nature, 222, 960--962.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....can they be segmented into the individual assemblies The answer to this problem is provided by the strong mutual excitation between the neurons belonging to the same assembly, probably acquired by Hebbian learning or auto association. This has been worked out in several rather theoretical papers ([Willshaw et al. 1969], Braitenberg, 1978] Palm, 1980] Palm, 1982] Hop eld, 1982] Palm, 1987] Palm, 1990] which essentially show that by controlling the total activity within a certain region of the cortex, for example by unspeci c inhibition, the superposition problem, i.e. the activation of two ....

....we used sizes of 40 40 for N , the correlation kernel K was circular (diameter 5, symmetric Gaussian with s.d. 2) and temporal correlation was determined by N = 20ms. 2. 3 Area C We modeled the central visual area C as a spiking variant of an Willshaw associative memory (e.g. [Willshaw et al. 1969], Palm, 1980] Palm, 1985] Palm, 1986] Sommer and Palm, 1999] similar as in the model of [Wennekers and Palm, 1997] In the following we describe brie y the structure and behavior of our model. A more detailed description and simulations of this type of associative memory can be found in ....

[Article contains additional citation context not shown here]

Willshaw, D., Buneman, O., and Longuet-Higgins, H. (1969). Nonholographic associative memory. Nature, 222:960-962. 28 type  e  i  x E e E i g 0 1 R a R r  h H e.s. 5ms 7ms 10ms 80mV 10mV 100 10mV 2ms 3:0ms 150ms 0:6mV i.s. 5ms 7ms 10ms 80mV 10mV 100 10mV 2ms 3:5ms - - i.g. 5ms 7ms 5ms 80mV 10mV 100 - - - - -

....results of this paper fit very well with computer simulations. Keywords: Artificial neural systems, matrix memory, correlation matrix, retrieval, storage. 1 Introduction The first works on associative matrix memories or correlation matrix memories appeared in the literature in the sixties [1, 2, 3] when the field of ANNs was still focused on arquitectures and learning rules for Perceptron [4] based networks. The first of these models to gain popularitywas the non linear matrix model [3] also known as Willshaw s model, due to its simplicity and efficient storage and retrieval properties. ....

....matrix memories or correlation matrix memories appeared in the literature in the sixties [1, 2, 3] when the field of ANNs was still focused on arquitectures and learning rules for Perceptron [4] based networks. The first of these models to gain popularitywas the non linear matrix model [3], also known as Willshaw s model, due to its simplicity and efficient storage and retrieval properties. In the early seventies, the linear matrix model was proposed simultaneously by Anderson [5] and Kohonen [6] Although this later model had also an efficient learning rule, crosstalk appeared ....

[Article contains additional citation context not shown here]

D.J. Willshaw, O.P. Buneman, and H.C. Longuet-Higgins. Non-holographic associative memory. Nature, 222, 1969.

....flat correlograms [8, 18] while phase coding models would predict modulated correlograms with non zero time lag of the central peak. In this work we try to reconciliate the original temporal correlation hypothesis with experimental results. First, we develop a new model of an associative memory [37, 23] which can be interpreted as one or several columns of a higher non sensory cortical area. We address the question how and where synchrony could be detected, i.e. how can the synchronized activation of two cell groups (presumably within 45ms) be read out by cortical circuits despite relatively ....

....on a slower time scale through a kind of self generated attention switching as an emergent network property of our model. 2 Spiking Associative Memory A variety of associative memory models has been introduced to demonstrate the capability of neural networks to learn and retrieve patterns (e.g. [37, 23, 13, 14]) Often these models use continuous variables that are interpreted as firing rates, and they exhibit some problems when addressed with a superposition of several patterns. Normally, this will result in an possibly incomplete activation of a superposition of stored patterns. Wennekers and Palm ....

[Article contains additional citation context not shown here]

D.J. Willshaw, O.P. Buneman, and H.C. Longuet-Higgins. Non-holographic as- sociative memory. Nature, 222:960-962, 1969.

....Singer, 1996] while phase coding models would predict correlograms that are modulated in the gamma frequency range with non zero time lag of the central peak. In this work we try to reconcile the TCH with the experimental results. In a first step, we develop a new model of an associative memory [Willshaw et al. 1969, Palm, 1980, Knoblauch, 1999] which we interpret as one or several columns of a non sensory corti cal area. We demonstrate that through a symmetry between an excitatory and a certain inhibitory population, we obtain the capability of autonomous threshold control and fast pat tern separation ....

....and separation in the 7 presence of realistic synaptic and axonal delays investigating single retrievals initiated by a superposition of address patterns. Different models of associative memory were introduced to demonstrate the capability of neural networks to learn and retrieve patterns (e.g. [Willshaw et al. 1969], Palm, 1980] Hopfield, 1982] Hopfield, 1984] Often, these models use continuous variables that are in terpreted as firing rates, and they exhibit some problems when addressed with a superposition of several patterns. Normally, this will result in an activation of a superposition of stored ....

[Article contains additional citation context not shown here]

Willshaw, D., Buneman, O., and Longuet-Higgins, H. (1969). Non- holographic associative memory. Nature, 222:960-962. 35

....compared, and otherwise processed. One candidate theory of higher level cognitive processing begins with the ideas that concepts can be represented as distributed patterns of neuronal activity, and that relationships between concepts can be represented as associations in an associative memory (Willshaw, Buneman, and Longuet Higgins 1969; Hinton 1989; Rumelhart, McClelland, and the PDP research group 1986) However, when we consider the requirement of higher level processing that relationships themselves must be examinable, a problem immediately arises: knowledge about relationships is hidden in the weights of the network and is ....

....like those in Figure 1 to be solved. Pairwise associations can be learned by many types of neural networks, including feedforward networks trained via backpropagation (slow learning) Rumelhart, Hinton, and Williams 1986) and hetero associative memory networks (one shot learning) such as Willshaw, Buneman, and Longuet Higgins s (1969) associative network. In both feedforward and associative networks, the knowledge about associations can be #### to produce one elementofapairgiven the other, but cannot be #####################. In order for knowledge to be manipulated, e.g. passed to other networks for further processing, it ....

[Article contains additional citation context not shown here]

Willshaw, D. J., O. P. Buneman, and H. C. Longuet-Higgins (1969). Non-holographic associative memory. ###### ###, 960-962.

....are possible, since switches can only ever be turned on. Indeed, the mean number of spurious errors increases as more pattern pairs are learnt. With palimpsest schemes however, where switches can be turned off, both types of error can occur. 3.3 Theoretical Capacity 3.3. 1 Hetero association [Willshaw et al. 69] derive conditions for efficient use of the WN under heteroassociation, by attempting to keep the number of spurious errors in retrieved patterns small. Consider the learning of R random pattern pairs. The probability that a particular switch, w ij , has been triggered at some time during the ....

D Willshaw, O Buneman, and H LonguetHiggins. Non-holographic associative memory. Nature, 222, 1969.

....given that they have a synaptic contact in one direction. In our model the synaptic transmission efficacy is formed in a Hebbian learning phase preceding the retrieval trials. The learning is modeled as simplistic as possible: We use the clipped synaptic modification rule of the Willshaw model [Willshaw et al. 1969], driven by a set of on off activity configurations (memory patterns) presented to the network. The configurations are random and overlapping, i.e. each contains a fixed number, k, of active cells, and each cell can be active in more than one memory pattern. The specification of the investigated ....

....i.e. each contains a fixed number, k, of active cells, and each cell can be active in more than one memory pattern. The specification of the investigated memory tasks (pattern number, P , and sparseness, p) was oriented on the efficient operation range of abstracted sparse associative memories [Willshaw et al. 1969, Palm and Sommer, 1995, Schwenker et al. 1996, Sommer and Palm, 1999] We used memory patterns with k = 10 active neurons. On these tasks we could directly compare the retrieval performance with theory and simulation results of the original and the feedback Willshaw model. Since only k = 10 ....

Willshaw, D. J., Buneman, O. P., and Longuet-Higgins, H. C. (1969). Nonholographic associative memory. Nature, 222:960--962.

....Fig. 1. ANN model of LTD based learning (see text for explanation) 2 The Model We compare the pattern recognition performance of the multi compartmental Purkinje cell model with the performance of an arti cial neural network (ANN) The ANN used is a modi ed version of an associative net [10] with real valued synapses and an LTD learning rule (Fig. 1) During the learning phase, each binary input pattern is stored in the ANN by decreasing the weights of all synapses that receive input by a factor of 0.5. During the recall phase, the response of the ANN to a pattern is given by the sum ....

D. J. Willshaw, O. P. Buneman, H. C. Longuet-Higgins, Non-holographic associative memory, Nature 222 (1969) 960-962. 6

.... Theta Gamma P ff r ff i h ff j Delta , where c ij is a binary variable indicating the existence of the j to i connection, and Theta(x) 1 if x 0 and 0 else. Similarly, the weights from i 2 R neurons to j 2 H neurons are set. This is the Hebbian rule first studied by Willshaw et al. [21]. Once the weights are set, the M pairs of patterns are stored as attractors, or fixed points in the 2 layer network. The dendritic sums for neurons i 2 R and j 2 H at time t are: h i (t) g NH X j=1 w ij x j (t Gamma 1) NS X k=1 w ik x k (t) h j (t) NR X i=1 w ji x i (t) 1) ....

D. Willshaw, O.P. Buneman, and H.C. Longuet-Higgins. Non-holographic associative memory. Nature, 222:960--962, 1969.

....However, the values of the connections are set as follows. For neuron j in A and i in B, the weight, w ij , from j to i is set to 1 if j 2 G A k and i 2 G B k for any k = 1; 2; 3; N . Otherwise, the weight is set to 0. This is a Hebbian style learning rule first studied by Willshaw et al. [12]. The weights from B to A are set in the same way. In addition, layer B also projects a global inhibition onto all neurons of layer A. The net effect is to make each neuron group, G A=B k , an attractor for the A Gamma B system, which would be completely activated if the attractor were to be ....

D. Willshaw, O. Buneman, and H. Longuet-Higgins. Non-holographic associative memory. Nature, 222:960-- 962, 1969.

....Neural Information Processing University of Ulm, 89069 Ulm, Germany fsommer,palmg informatik.uni ulm.de Abstract Similarity based fault tolerant retrieval in neural associative memories (NAM) has not lead to wiedespread applications. A drawback of the efficient Willshaw model for sparse patterns [Ste61, WBLH69], is that the high asymptotic information capacity is of little practical use because of high cross talk noise arising in the retrieval for finite sizes. Here a new bidirectional iterative retrieval method for the Willshaw model is presented, called crosswise bidirectional (CB) retrieval, ....

....cells in many places of the cortex. An important property of a NAM model is its information capacity, measuring how efficient the synaptic weights are used. In the early sixties Steinbuch realized under the name Lernmatrix a memory model with binary synapses which is now known as Willshaw model [Ste61, WBLH69]. The great variety of NAM models proposed since then, many triggered by Hopfield s work [Hop82] do not reach the high asymptotic information capacity of the Willshaw model. For finite network size, the Willshaw model does not optimally retrieve the stored information, since the inner product ....

D. J. Willshaw, O. P. Buneman, and H. C. Longuet-Higgins. Nonholographic associative memory. Nature, 222:960--962, 1969.

....architecture, as demonstrated by the experiments of sections 3 and 5. Section 7 concludes this paper with a short summary of the main results of this work. 2 Learning Model The development of the DRAMA architecture was first inspired from the model of associative memory proposed by Willshaw [42], which is an abstract model of the hippocampus. Its development was driven by our wish to build a control architecture to enable real time control and learning in a physical autonomous agent. In particular, the choice of using a connectionist model and especially a Hebbian associative memory was ....

....environmental constraints 1 , 3) as little built in knowledge as possible to keep the system unspecific to a particular type of implementation (task, agent or environment) 2. 1 The Willshaw net The original version of the Willshaw net was developed as a model of biological associative memory [42]. It can be thought of as a fully connected network with symmetrical connections, whose weights are updated following a basic Hebbian rule, i.e. only the weights of connections with co active nodes are reinforced. The patterns consist of pairs of input output bit strings. The patterns are ....

Willshaw D., Buneman O., Longuet-Higgins H.", (1969), `Non-holographic associative memory', Nature, vol. 222, pp. 960-962.

....DRAMA architecture, as demonstrated by the experiments of Sections 3 and 5. Section 7 concludes this paper with a short summary of the main results of this work. 2 LEARNING MODEL The development of the DRAMA architecture was first inspired from the model of associative memory proposed by Willshaw (1969), which is an abstract model of the hippocampus. Its development was driven by our wish to build a control architecture to enable real time control and learning in a physical autonomous agent. In particular, the choice of using a connectionist model and especially a Hebbian associative memory ....

....constraints, 1 3) as little built in knowledge as possible to keep the system unspecific to a particular type of implementation (task, agent or environment) 2. 1 The Willshaw net The original version of the Willshaw net was developed as a model of biological associative memory DRAMA 37 (Willshaw et al. 1969). It can be thought of as a fullyconnected network with symmetrical connections, whose weights are updated following a basic Hebbian rule, i.e. only the weights of connections with co active nodes are reinforced. The patterns consist of pairs of inputoutput bit strings. The patterns are presented ....

Willshaw, D. Buneman, O. & Longuet-Higgins, H. (1969). Non-holographic associative memory. Nature, 222, 960-962.

.... of fundamental memories [13] Evidently, the information storage capacity (number of bits which can be stored and recalled associatively) of the morphological auto associative memory also exceeds the respective number for certain linear matrix associative memories which was calculated by Palm [15, 23]. The discrete correlation recorded Hopfield net is one of the most popular methods for auto association of binary patterns [7, 8, 1, 12] Unlike the Hopfield network, which is a recurrent neural network, our model provides the final result in one pass through the network without any significant ....

D.J. Willshaw, O.P. Buneman, and H.C. Longuet-Higgins. Non-holographic associative memory. Nature, 222:960--962, 1969.

....Interneuron plasticity in associative networks Hajime Hirase y Michael Recce Department of Anatomy and Developmental Biology University College London London WC1E 6BT, UK email: fhhirase,recceg anat.ucl.ac. uk Introduction The associative memory, proposed by Willshaw and co workers[2] consists of binary valued neurons, with binary valued, clipped synapses. Memory events are stored by extrinsically activating principal cells, the number of active cells in any one event is constant, and the set of active cells is selected randomly (i.e. maximising the storage capacity) Recall ....

D. J. Willshaw, O. P. Buneman, and Longuet-Higgins. Non-holographic associative memory. Nature (Lond), 222:960--962, 1969.

....based on a descent biophysical model. 3 Lessons from neural associative networks 3.1 Conventional associative networks Binary associative networks have been proposed as quite abstract computational models for cortical neuron networks. The most prominent text book examples are the Willshaw model [78], or the Little Hopfield model [35, 27] In these models the underlying biophysical description of a neuron is that of a binary threshold device. A binary network state at any fixed time, i.e. an activity pattern, is interpreted as a distributed representation of an information entity. The ....

....studies with experimental data, see for instance [4, 5] but the correspondence is vague. Here we will use the results in another way: we select those models fulfilling the efficiency requirement and use their properties as constraints in a model with closer biophysical faithfulness. Willshaw [78] calculated the information efficiency of a binary associative memory with Hebbian 5 learning. Hebbian learning of a single neuronal configuration, i.e. a pattern pair, imprints the outerproduct matrix of the pattern pair into the synaptic coupling matrix [41] For several configurations Willshaw ....

D. J. Willshaw, O. P. Buneman, and H. C. Longuet-Higgins. Non-holographic associative memory. Nature, 222:960--962, 1969.

....incomplete input pattern. The output threshold function, t , is set to the level equal to the number of bits set in the input (termed fixed thresholding, see [3] for details) To match a proportion of the input pattern the threshold is reduced. An alternative is to use L max threshold method [5], which operates by selecting the L highest responding output neurons and setting these to 1, while all others are set to zero. The advantage of this method is that it is independent of the characteristics of the input pattern, but does require that all trained separators have a fixed bit ....

D.J. Willshaw, O.P. Buneman, H.C. Longuet-Higgins, "Non-holographic associative memory", Nature, Vol. 222, 1969, p960-962.

....discovery activities. 6.3 Models of Memory Storage In this section we provide an outline of the various biologically inspired computerised models connected with memory storage in the brain. Knoblauch and Palm (2001) 42] took a similar approach to autoassociative networks as by Willshaw [79, 75, 78, 76, 80, 77] and extend it based on biological neurons and synapses. In particular Knoblauch and Palm added characteristics that represent the spiking actions of real neurons in addition to the characteristics of spatio temporal integration on dendrites. Individual cells are modelled like spiking neurons : ....

D. Willshaw, O. Buneman, and H. Longuet-Higgins. Nonholographic associative memory. Nature, 222:960-962, 1969.

No context found.

Willshaw, D. J., Buneman, & Longuet-Higgins, H. C. (1969). Nonholographic associative memory. Nature, 222, 960--962.

No context found.

D.J. Willshaw, O.P. Buneman, and H.C. Longuet-Higgins. Non-holographic associative memory. Nature, 222(7):960--962, Jun 1969.

No context found.

O P Buneman D J Willshaw and H C Longuet-Higgins. Non holographic associative memory. Nature, June 1969.

No context found.

Willshaw, D., Buneman, O., and Longuet-Higgins, H. (1969). Nonholographic associative memory. Nature, 222:960-962. 21

No context found.

D. Willshaw, O.P. Buneman and H.C. Longuet-Higgins. Non-holographic associative memory. Nature 222:960-962.

No context found.

Willshaw D., Buneman O. P. and Longuet-Higgins H. (1969) NonHolographic Associative Memory. Nature Vol 222 pp. 960-962.

No context found.

D. J. Willshaw, O. P. Buneman, H. C. Longuet-Higgins, "Non-Holographic Associative Memory", 1969, Nature, p 222, June 7, Vol. 9.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC