After classically conditioned learning, dopaminergic cells in the substantia nigra pars compacta (SNc) respond immediately to unexpected conditioned stimuli (CS) but omit formerly seen responses to expected unconditioned stimuli, notably rewards. These cells play an important role in reinforcement learning. A neural model explains the key neurophysiological properties of these cells before, during, and after conditioning, as well as related anatomical and neurophysiological data about the pedunculopontine tegmental nucleus (PPTN), lateral hypothalamus, ventral striatum, and striosomes. The model proposes how two parallel learning pathways from limbic cortex to the SNc, one devoted to excitatory conditioning (through the ventral striatum, ventral pallidum, and PPTN) and the other to adaptively timed inhibitory conditioning (through the striosomes), control SNc responses. The excitatory pathway generates CS-induced excitatory SNc dopamine bursts. The inhibitory pathway prevents dopamine bursts in response to Humans and animals can learn to predict both the amounts and times of expected rewards. The dopaminergic cells of the substantia nigra pars compacta (SNc) have unique firing patterns related to the predicted and actual times of reward (Ljungberg et

We propose a systems-level computational model of the basal ganglia based closely on known anatomy and physiology. First, we assume that the thalamic targets, which relay ascending information to cortical action and planning areas, are tonically inhibited by the basal ganglia. Second, we assume that the output stage of the basal ganglia, the internal segment of the globus pallidus (GPi), selects a single action from several competing actions via lateral interactions. Third, we propose that a form of local working memory exists in the form of reciprocal connections between the external globus pallidus (GPe) and the subthalamic nucleus (STN). As a test of the model, the system was trained to learn a sequence of states that required the context of previous actions. The striatum, which was assumed to represent a conjunction of cortical states, directly selected the action in the GP during training. The STN-to-GP connection strengths were modi�ed by an associative learning

In this article we compare methods for temporal sequence learning (TSL) across the disciplines machine-control, classical conditioning, neuronal models for TSL as well as spiketiming dependent plasticity. This review will briefly introduce the most influential models and focus on two questions: 1) To what degree are reward-based (e.g. TD-learning) and correlation based (hebbian) learning related? and 2) How do the different models correspond to possibly underlying biological mechanisms of synaptic plasticity? We will first compare the different models in an open-loop condition, where behavioral feedback does not alter the learning. Here we observe, that reward-based and correlation based learning are indeed very similar. Machine-control is then used to introduce the problem of closed-loop control (e.g. “actor-critic architectures”). Here the problem of evaluative (“rewards”) versus nonevaluative (“correlations”) feedback from the environment will be discussed showing that both learning approaches are fundamentally different in the closed-loop condition. In trying to answer the second question we will compare neuronal versions of the different learning architectures to the anatomy of the involved brain structures (basal-ganglia, thalamus and

Synaptic runaway denotes the formation of erroneous synapses and premature functional decline accompanying activity-dependent learning in neural networks. This work studies synaptic runaway both analytically and numerically in binary-firing associative memory networks. It turns out that synaptic runaway is of fairly moderate magnitude in these networks under normal, baseline, conditions. However, it may become extensive if the threshold for Hebbian learning is reduced. These findings are combined with recent evidence for arrested N-methyl-D-aspartate (NMDA) maturation in schizophrenics, to formulate a new hypothesis concerning the pathogenesis of schizophrenic psychotic symptoms, in neural terms. Supported by an Alon Fellowship, and to whom correspondence should be addressed. 1 Introduction Learning in associative neural networks has generated a considerable amount of interest as a possible model of memory and learning in the brain [1, 2, 3, 4, 5]. During learning, previously stor...

You might find this additional information useful... This article cites 92 articles, 48 of which you can access free at:

We implement and study a computational model of Stevens' [1992] theory of the pathogenesis of schizophrenia. This theory hypothesizes that the onset of schizophrenia is associated with reactive synaptic regeneration occurring in brain regions receiving degenerating temporal lobe projections. Concentrating on one such area, the frontal cortex, we model a frontal module as an associative memory neural network whose input synapses represent incoming temporal projections. Modeling Stevens' hypothesized pathological synaptic changes in this framework results in adverse side effects reminiscent of hallucinations and delusions seen in schizophrenia: spontaneous, stimulusindependent retrieval of stored memories focused on just a few of the stored patterns. These could account for the occurrence of schizophrenic delusions and hallucinations without any apparent external trigger, and for their tendency to concentrate on a few central cognitive and perceptual themes. The model explains why schizo...

This paper presents the hypothesis that NMDA receptor delayed maturation (NRDM) may lead to the pathogenesis of schizophrenic psychotic symptoms. This hypothesis is further analyzed in the language of a neural modeling formulation. This formulation points to a possible chain of pathological events, leading from molecular-level NRDM to over-increased synaptic plasticity, and to the formation of pathological attractors, a putative macroscopic-level correlate of schizophrenic positive symptoms. The relations of the NRDM hypothesis to other alterations which are assumed to take place in schizophrenia are discussed, together with possible ways to test this hypothesis. 1 The NRDM Hypothesis NMDA receptor antagonist drugs exacerbate schizophrenic symptoms and, in animals, produce neurodegenerative effects in brain areas thought be to involved in the pathogenesis of this disorder. Consequently, it has been proposed that spontaneously occurring NMDA receptor hypofunction causes schizophrenia...

Endocannabinoid–dopamine interactions in striatal