We explore a computationally efficient method of simulating realistic networks of neurons introduced by Knight, Manin, and Sirovich (1996) in which integrate-and-fire neurons are grouped into large populations of similar neurons. For each population, we form a probability density which represents the distribution of neurons over all possible states. The populations are coupled via stochastic synapses in which the conductance of a neuron is modulated according to the firing rates of its presynaptic populations. The evolution equation for each of these probability densities is a partial differential-integral equation which we solve numerically. Results obtained for several example networks are tested against conventional computations for groups of individual neurons. We apply this approach to modeling orientation tuning in the visual cortex. Our population density model is based on the recurrent feedback model of a hypercolumn in cat visual cortex of Somers et al. (1995). We simulate the response to oriented flashed bars. As in the Somers model, a weak orientation bias provided by feed-forward lateral geniculate input is transformed by intracortical circuitry into sharper orientation tuning which is independent of stimulus contrast. The population density approach appears to be a viable method for simulating large neural networks. Its computational efficiency overcomes some of the restrictions imposed by computation time in individual

We demonstrate that single-variable integrate-and-fire models can quantitatively capture the dynamics of a physiologically-detailed model for fast-spiking cortical neurons. Through a systematic set of approximations, we reduce the conductance based model to two variants of integrate-and-fire models. In the first variant (non-linear integrate-and-fire model), parameters depend on the instantaneous membrane potential whereas in the second variant, they depend on the time elapsed since the last spike (Spike Response Model). The direct reduction links features of the simple models to biophysical features of the full conductance based model. To quantitatively

Introduction Neuronal activity due to recurrent excitations in the form of a spatially localized pulse or bump has been proposed as a mechanism for feature selectivity in models of the visual system (Somers, Nelson, & Sur, 1995; Hansel & Sompolinsky, 1998), the head direction system (Skaggs, Knieram, Kudrimoti, & McNaughton, 1995; Zhang, 1996; Redish, Elga, & Touretzky, 1996), and working memory (Wilson & Cowan, 1973; Amit & Brunel, 1997; Camperi & Wang, 1998). Many of the previous mathematical formulations of such structures have employedpopulation rate models (Wilson &Cowan, 1972, 1973; Amari, 1977; Kishimoto & Amari, 1979; Hansel & Sompolinsky, 1998). (See Ermentrout, 1998, for a recent review.) Here, we consider a network of spiking neurons that shows such structures and investigate their properties. In our network we #nd localized time-stationary states

We analyze the effect of noise in integrate-and-fire neurons driven by timedependent input, and compare the diffusion approximation for the membrane potential to escape noise. It is shown that for time-dependent sub-threshold input, diffusive noise can be replaced by escape noise with a hazard function that has a Gaussian dependence upon the distance between the (noise-free) membrane voltage and threshold. The approximation is improved if we add to the hazard function a probability current proportional to the derivative of the voltage. Stochastic resonance in response to periodic input occurs in both noise models and exhibits similar characteristics.

The prevalence of coherent oscillations in various frequency ranges in the central nervous system raises the question of the mechanisms that synchronize large populations of neurons. We study synchronization in models of large networks of spiking neurons with random sparse connectivity. Synchrony occurs only when the average number of synapses, M , that a cell receives is larger than a critical value, M c . Below M c , the system is in an asynchronous state. In the limit of weak coupling, assuming identical neurons, we reduce the model to a system of phase oscillators which are coupled via an effective interaction, \Gamma. In this framework, we develop an approximate theory for sparse networks of identical neurons to estimate M c analytically from the Fourier coefficients of \Gamma. Our approach relies on the assumption that the dynamics of a neuron depend mainly on the number of cells that are presynaptic to it. We apply this theory to compute M c for a model of inhibitory networks of integrate-and-fire (I&F) neurons as a function of the intrinsic neuronal properties (e.g., the refractory period T r ), the synaptic time constants and the strength of the external stimulus, I ext . The number M c is found to be non-monotonous with the strength of I ext . For T r = 0, we estimate the minimum value of M c over all the parameters of the model to be 363:8. Above M c , the neurons tend to fire in: 1) smeared one cluster states at high firing rates and 2) smeared two or more cluster states at low firing rates. Refractoriness decreases M c at intermediate and high firing rates. These results are compared against numerical simulations. We show numerically that systems with different sizes, N , behave in the same way provided the connectivity, M , is such a way that 1=M eff = 1=...

We study analytically a model of long-term synaptic plasticity where synaptic changes are triggered by presynaptic spikes, postsynaptic spikes, and the time dierences between presynaptic and postsynaptic spikes. The changes due to correlated input and output spikes are quanti- ed by means of a learning window. We show that plasticity can lead to an intrinsic stabilization of the mean ring rate of the postsynaptic neuron. Subtractive normalization of the synaptic weights (summed over all presynaptic inputs converging on a postsynaptic neuron) follows if, in addition, the mean input rates and the mean input correlations are identical at all synapses. If the integral over the learning window is positive, ring-rate stabilization requires a non-Hebbian component, whereas such a component is not needed, if the integral of the learning window is negative. A negative integral corresponds to `anti-Hebbian' learning in a model with slowly varying ring rates. For spike-based learning, a strict distinction between Hebbian and `anti-Hebbian' rules is questionable since learning is driven by correlations on the time scale of the learning window. The correlations between presynaptic and postsynaptic ring are evaluated for a piecewise-linear Poisson model and for a noisy spiking neuron model with refractoriness. While a negative integral over the learning window leads to intrinsic rate stabilization, the positive part of the learning window picks up spatial and temporal correlations in the input.

Human cognition is unique in the way in which it relies on combinatorial (or compositional) structures. Language provides ample evidence for the existence of combinatorial structures, but they can also be found in visual cognition. To understand the neural basis of human cognition, it is therefore essential to understand how combinatorial structures can be instantiated in neural terms. In his recent book on the foundations of language, Jackendoff formulated four fundamental problems for a neural instantiation of combinatorial structures: the massiveness of the binding problem, the problem of 2, the problem of variables and the transformation of combinatorial structures from working memory to long-term memory. This paper aims to show that these problems can be solved by means of neural ‘blackboard ’ architectures. For this purpose, a neural blackboard architecture for sentence structure is presented. In this architecture, neural structures that encode for words are temporarily bound in a manner that preserves the structure of the sentence. It is shown that the architecture solves the four problems presented by Jackendoff. The ability of the architecture to instantiate sentence structures is illustrated with examples of sentence complexity observed in human language performance. Similarities exist between the architecture for sentence structure and blackboard architectures for combinatorial structures in visual cognition, derived from the structure of the visual cortex. These architectures are briefly discussed, together with an example of a combinatorial structure in which the blackboard architectures for language and vision are combined. In this way, the architecture for language is grounded in perception. 2 Content

Abstract: Many biological neural network models face the problem of scalability because of the limited computational power of today’s computers. Thus, it is difficult to assess the efficiency of these models to solve complex problems such as image processing. Here, we describe how this problem can be tackled using event-driven computation. Only the neurons that emit a discharge are processed and, as long as the average spike discharge rate is low, millions of neurons and billions of connections can be modeled. We describe the underlying computation and implementation of such a mechanism in SpikeNET, our neural network simulation package. The type of model one can build is not only biologically compliant, it is also computationally efficient as 400 000 synaptic weights can be propagated per second on a standard desktop computer. In addition, for large networks, we can set very small time steps (less than 0.01 ms) without significantly increasing the computation time. As an example, this method is applied to solve complex cognitive tasks such as face recognition in natural images.

The width of the orientation tuning curves of the spike response of neurons in V1 is invariant to contrast. This property constrains the possible mechanisms underlying orientation selectivity. It has been suggested that noise circumvents the iceberg effect that would prevent contrast invariance in the purely feedforward mechanism. Here we investigate systematically how noise contributes to the contrast invariance of orientation tuning curves in V1. We study three models of increasing complexity: a simple threshold-linear firing rate model, a leaky integrateand-fire model, and a conductance-based model. We show that the noise transmutes the threshold nonlinearity of the input–output relationships into an approximate power law without a threshold within some firing rate range. This implies that, under certain conditions which are derived here, the tuning of the neuron output is approximately contrast invariant. In particular we show that this mechanism for contrast invariance requires that the neuron firing rate must not be too large and that increasing or lowering the contrast too much destroys this invariance. We also show that if this mechanism operates in V1, the spike response, R, and average voltage response V of the neurons in V1 should vary with the contrast, C, according to R(C) � V(C) �. The exponent � can be estimated from the amount by which the spike tuning curve is sharpened with respect to the voltage tuning curves of the neurons. This prediction does not depend on the specifics of the model and can be tested experimentally. Key words: orientation selectivity; primary visual cortex; V1; contrast invariance; noise; integrate-and-fire model; conductance-based model

There is growing evidence from psychophysical and neurophysiological studies that the brain utilizes Bayesian principles for inference and decision making. An important open question is how Bayesian inference for arbitrary graphical models can be implemented in networks of spiking neurons. In this paper, we show that recurrent networks of noisy integrate-and-fire neurons can perform approximate Bayesian inference for dynamic and hierarchical graphical models. The membrane potential dynamics of neurons is used to implement belief propagation in the log domain. The spiking probability of a neuron is shown to approximate the posterior probability of the preferred state encoded by the neuron, given past inputs. We illustrate the model using two examples: (1) a motion detection network in which the spiking probability of a direction-selective neuron becomes proportional to the posterior probability of motion in a preferred direction, and (2) a two-level hierarchical network that produces attentional effects similar to those observed in visual cortical areas V2 and V4. The hierarchical model offers a new Bayesian interpretation of attentional modulation in V2 and V4. 1