Several formulations of correlation-based Hebbian learning are reviewed. On the presynaptic side, activity is described either by a firing rate or by presynaptic spike arrival. The state of the postsynaptic neuron can be described by its membrane potential, its firing rate, or the timing of backpropagating action potentials (BPAPs). It is shown that all of the above formulations can be derived from the point of view of an expansion. In the absence of BPAPs potentials, it is natural to correlate presynaptic spikes with the postsynaptic membrane potential. Time windows of spike time dependent plasticity arise naturally, if the timing of postsynaptic spikes is available at the site of the synapse as it is the case in the presence of BPAPs. With an appropriate choice of parameters, Hebbian synaptic plasticity has intrinsic normalization properties that stabilizes postsynaptic firing rates and leads to subtractive weight normalization.

Spiking neurons are very flexible computational modules, which can implement with different values of their adjustable synaptic parameters an enormous variety of different transformations F from input spike trains to output spike trains. We examine in this article the question to what extent a spiking neuron with biologically realistic models for dynamic synapses can be taught via spike-timing-dependent plasticity (STDP) to implement a given transformation F. We consider a supervised learning paradigm where during training the output of the neuron is clamped to the target signal (teacher forcing). The well-known Perceptron Convergence Theorem asserts the convergence of a simple supervised learning algorithm for drastically simplified neuron models (McCulloch-Pitts neurons). We show that in contrast to the Perceptron Convergence Theorem no theoretical guarantee can be given for the convergence of STDP with teacher forcing that holds for arbitrary input spike patterns. On the other hand we prove that average case versions of the

In this article we compare methods for temporal sequence learning (TSL) across the disciplines machine-control, classical conditioning, neuronal models for TSL as well as spiketiming dependent plasticity. This review will briefly introduce the most influential models and focus on two questions: 1) To what degree are reward-based (e.g. TD-learning) and correlation based (hebbian) learning related? and 2) How do the different models correspond to possibly underlying biological mechanisms of synaptic plasticity? We will first compare the different models in an open-loop condition, where behavioral feedback does not alter the learning. Here we observe, that reward-based and correlation based learning are indeed very similar. Machine-control is then used to introduce the problem of closed-loop control (e.g. “actor-critic architectures”). Here the problem of evaluative (“rewards”) versus nonevaluative (“correlations”) feedback from the environment will be discussed showing that both learning approaches are fundamentally different in the closed-loop condition. In trying to answer the second question we will compare neuronal versions of the different learning architectures to the anatomy of the involved brain structures (basal-ganglia, thalamus and

In timing-based neural codes, neurons have to emit action potentials at precise moments in time. We use a supervised learning paradigm to derive a synaptic update rule that optimizes by gradient ascent the likelihood of postsynaptic firing at one or several desired firing times. We find that the optimal strategy of up- and downregulating synaptic efficacies depends on the relative timing between presynaptic spike arrival and desired postsynaptic firing. If the presynaptic spike arrives before the desired postsynaptic spike timing, our optimal learning rule predicts that the synapse should become potentiated. The dependence of the potentiation on spike timing directly reflects the time course of an excitatory postsynaptic potential. However, our approach gives no unique reason for synaptic depression under reversed spike timing. In fact, the presence and amplitude of depression of synaptic efficacies for reversed spike timing depend on how constraints are implemented in the optimization problem. Two different constraints, control of postsynaptic rates and control of temporal locality, are studied. The relation of our results to spike-timing-dependent plasticity and reinforcement learning is discussed.

Synaptic plasticity is thought to be the neuronal correlate of learning. Moreover, modification of synapses contributes to the activity-dependent homeostatic maintenance of neurons and neural networks. In this chapter, we review theories of synaptic plasticity and show that both homeostatic control of activity and detection of correlations in the presynaptic input can arise from spike-timing dependent plasticity (STDP). Relations to classical rate-based Hebbian learning are discussed.

www.his.sunderland.ac.uk We present an approach for recognition and clustering of spatio temporal patterns based on networks of spiking neurons with active dendrites and dynamic synapses. We introduce a new model of an integrate-andfire neuron with active dendrites and dynamic synapses (ADDS) and its synaptic plasticity rule. The neuron employs the dynamics of the synapses and the active properties of the dendrites as an adaptive mechanism for maximizing its response to a specific spatio-temporal distribution of incoming action potentials. The learning algorithm follows recent biological evidence on synaptic plasticity. It goes beyond the current computational approaches which are based only on the relative timing between single preand post-synaptic spikes and implements a functional dependence based on the state of the dendritic and somatic membrane potentials around the pre- and post-synaptic action potentials. The learning algorithm is demonstrated to effectively train the neuron towards a selective response determined by the spatio-temporal pattern of the onsets of input spike trains. The model is used in the implementation of a part of a robotic system for natural language instructions. We test the model with a robot whose goal is to recognize and execute language instructions. The research in this article demonstrates the potential of spiking neurons for processing spatio-temporal patterns and the experiments present spiking neural networks as a paradigm which can be applied for modeling sequence detectors at word level for robot instructions. Key words: spiking neurons, active dendrites, dynamic synapses, synaptic plasticity, temporal sequence detection, natural language, intelligent robotics 1 1

You might find this additional information useful... This article cites 34 articles, 15 of which you can access free at:

Spike timing-dependent synaptic plasticity (STDP) has emerged as the preferred framework linking patterns of pre- and postsynaptic activity to changes in synaptic strength. Although synaptic plasticity is widely believed to be a major component of learning, it is unclear how STDP itself could serve as a mechanism for general purpose learning. On the other hand, algorithms for reinforcement learning work on a wide variety of problems, but lack an experimentally established neural implementation. Here, we combine these paradigms in a novel model in which a modified version of STDP achieves reinforcement learning. We build this model in stages, identifying a minimal set of conditions needed to make it work. Using a performance-modulated modification of STDP in a two-layer feedforward network, we can train output neurons to generate arbitrarily selected spike trains or population responses. Furthermore, a given network can learn distinct responses to several different input patterns. We also describe in detail how this model might be implemented biologically. Thus, our model offers a novel and biologically plausible implementation of reinforcement learning that is capable of training a neural population to produce a very wide range of possible mappings between synaptic input and spiking output.

In standard neural network models neurons are described in terms of mean firing rates, viz., an analog signal. Most real neurons, however, communicate by pulses, called action potentials or simply `spikes'. In this chapter the main di#erences between spike coding and rate coding are described. The integrate-and-fire model is studied as a simple model of a spiking neuron. Fast transients, synchrony, and coincidence detection are discussed as examples where spike coding is relevant. A description by spikes rather than rates has implications for learning rules. We show the relation of a spike-time dependent learning rule to standard Hebbian learning. Finally, learning rule and temporal coding are illustrated using the example of a coincidence detecting neuron in the barn owl auditory system. Keywords: temporal coding, coincidence detection, spikes, spiking neurons, integrateand -fire neurons, auditory system, Hebbian learning, spike-time dependent plasticity 1. SPIKES AND RATES In mos...

We studied the hypothesis that synaptic dynamics is controlled by three basic principles: (1) synapses adapt their weights so that neurons can effectively transmit information, (2) homeostatic processes stabilize the mean firing rate of the postsynaptic neuron, and (3) weak synapses adapt more slowly than strong ones, while maintenance of strong synapses is costly. Our results show that a synaptic update rule derived from these principles shares features, with spike-timing-dependent plasticity, is sensitive to correlations in the input and is useful for synaptic memory. Moreover, input selectivity (sharply tuned receptive fields) of postsynaptic neurons develops only if stimuli with strong features are presented. Sharply tuned neurons can coexist with unselective ones, and the distribution of synaptic weights can be unimodal or bimodal. The formulation of synaptic dynamics through an optimality criterion provides a simple graphical argument for the stability of synapses, necessary for synaptic memory.